Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

repo cleanup and maven central release #6

Merged
merged 3 commits into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file removed .DS_Store
Binary file not shown.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@

.idea
.java-version
/target
3 changes: 0 additions & 3 deletions .idea/.gitignore

This file was deleted.

22 changes: 0 additions & 22 deletions .idea/compiler.xml

This file was deleted.

7 changes: 0 additions & 7 deletions .idea/encodings.xml

This file was deleted.

45 changes: 0 additions & 45 deletions .idea/jarRepositories.xml

This file was deleted.

15 changes: 0 additions & 15 deletions .idea/misc.xml

This file was deleted.

124 changes: 0 additions & 124 deletions .idea/uiDesigner.xml

This file was deleted.

6 changes: 0 additions & 6 deletions .idea/vcs.xml

This file was deleted.

29 changes: 28 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,41 @@
### MAC Web Crawler
![Maven Central](https://img.shields.io/maven-central/v/cloud.anypoint/mule-web-crawler-connector)

**MAC Web Crawler** is a MuleSoft custom connector to provide web crawling capabilities to extract data from web pages subsequently based on the structure of the website.

Add this dependency to your application pom.xml

### Installation (using Cloud.Anypoint Dependency)

```xml
<dependency>
<groupId>cloud.anypoint</groupId>
<artifactId>mule-web-crawler-connector</artifactId>
<version>0.1.0</version>
<classifier>mule-plugin</classifier>
</dependency>
```

### Installation (building locally)

To use this connector, first [build and install](https://mac-project.ai/docs/mulechain-ai/getting-started) the connector into your local maven repository.
Then add the following dependency to your application's `pom.xml`:


```xml
<dependency>
<groupId>com.mule.mulechain</groupId>
<artifactId>mac-web-crawler</artifactId>
<version>0.1.0</version>
<classifier>mule-plugin</classifier>
</dependency>
```

### Installation into private Anypoint Exchange

You can also make this connector available as an asset in your Anyooint Exchange.

This process will require you to build the connector as above, but additionally you will need
to make some changes to the `pom.xml`. For this reason, we recommend you fork the repository.

Then, follow the MuleSoft [documentation](https://docs.mulesoft.com/exchange/to-publish-assets-maven) to modify and publish the asset.

70 changes: 34 additions & 36 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -8,56 +8,54 @@
<artifactId>mac-web-crawler</artifactId>
<version>0.1.0</version>
<packaging>mule-extension</packaging>
<name>mac-web-crawler Extension</name>
<name>MAC WebCrawler Connector</name>
<description>The MAC WebCrawler Connector enables a Mule application to crawl websites and retrieve content, potentially for creating vector embeddings for structured knowledge extraction.</description>
<url>https://mac-project.ai/docs/mac-webcrawler/connector-overview</url>
<licenses>
<license>
<name>MIT License</name>
<url>https://www.opensource.org/licenses/mit-license.php</url>
</license>
</licenses>
<developers>
<developer>
<name>Yogesh Mudaliar</name>
</developer>
<developer>
<name>Amir Khan</name>
<email>[email protected]</email>
<organization>Salesforce</organization>
<organizationUrl>https://www.mulesoft.com</organizationUrl>
</developer>
<developer>
<name>Mihael Bosnjak</name>
<email>[email protected]</email>
<organization>Salesforce</organization>
<organizationUrl>https://www.mulesoft.com</organizationUrl>
</developer>
<developer>
<name>Ryan Hoegg</name>
<email>[email protected]</email>
<organization>Hoegg Software, Co.</organization>
<organizationUrl>https://hoegg.software</organizationUrl>
</developer>
</developers>

<parent>
<groupId>org.mule.extensions</groupId>
<artifactId>mule-modules-parent</artifactId>
<version>1.1.3</version>
<version>1.3.2</version>
</parent>
<dependencies>
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.17.2</version>
<version>1.14.3</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.15.2</version>
</dependency>
</dependencies>
<repositories>
<repository>
<id>anypoint-exchange-v3</id>
<name>Anypoint Exchange V3</name>
<url>https://maven.eu1.anypoint.mulesoft.com/api/v3/maven</url>
<layout>default</layout>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>mulesoft-releases</id>
<name>mulesoft release repository</name>
<layout>default</layout>
<url>https://repository.mulesoft.org/releases/</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
<distributionManagement>
<snapshotRepository>
<id>exchange-repository</id>
<name>Exchange Repository</name>
<url>https://maven.anypoint.mulesoft.com/api/v1/organizations/9c7c42b3-1b21-4ec1-9183-047ec0921663/maven</url>
<layout>default</layout>
</snapshotRepository>
<repository>
<id>exchange-repository</id>
<name>Exchange Repository</name>
<url>https://maven.anypoint.mulesoft.com/api/v1/organizations/9c7c42b3-1b21-4ec1-9183-047ec0921663/maven</url>
<layout>default</layout>
</repository>
</distributionManagement>
</project>
Loading