MuleSoft WebCrawler Connector

MuleSoft AI Chain (MAC) Project

MAC Web Crawler

MuleSoft WebCrawler provides web crawling capabilities to extract data from web pages subsequently based on the structure of the website.

Requirements

• The maximum supported version for Java SDK is JDK 17. You can use JDK 17 only for running your application.
• Compilation with Java SDK must be done with JDK 11.

Installation (using maven central dependency)

<dependency>
   <groupId>io.github.mulesoft-ai-chain-project</groupId>
   <artifactId>mule4-webcrawler-connector</artifactId>
   <version>0.1.25</version>
   <classifier>mule-plugin</classifier>
</dependency>

Installation (building locally)

To use this connector, first build and install the connector into your local maven repository. Then add the following dependency to your application's pom.xml:

<dependency>
    <groupId>com.mulesoft.connectors</groupId>
    <artifactId>mule4-webcrawler-connector</artifactId>
    <version>0.1.25</version>
    <classifier>mule-plugin</classifier>
</dependency>

Installation into private Anypoint Exchange

You can also make this connector available as an asset in your Anyooint Exchange.

This process will require you to build the connector as above, but additionally you will need to make some changes to the pom.xml. For this reason, we recommend you fork the repository.

Then, follow the MuleSoft documentation to modify and publish the asset.