Download all Objects from an Amazon S3 Bucket using MuleSoft

Here’s a quick example on how you can download all the Objects from an Amazon S3 Bucket using the Amazon S3 Connector.  You can see the flow and the corresponding code (minus the HTTP and Amazon S3 Configuration) below.

<flow name="s3-exampleFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="/bucket" doc:name="HTTP" />
<s3:list-objects config-ref="Amazon_S3__Configuration" bucketName="s3-mule-bucket" doc:name="Amazon S3 - List Objects in Bucket" maxKeys="5" />
<foreach collection="#[payload.objectSummaries]" doc:name="For Each">
<set-variable variableName="fileKey" value="#[payload.getKey()]" doc:name="Store Filename" />
<s3:get-object-content config-ref="Amazon_S3__Configuration" bucketName="#[payload.getBucketName()]" key="#[payload.getKey()]" doc:name="Amazon S3 - Get Object" />
<object-to-byte-array-transformer doc:name="Object to Byte Array" />
<file:outbound-endpoint path="/Users/dejimjuang/Downloads" responseTimeout="10000" doc:name="Write File to Path" outputPattern="#[flowVars.fileKey] "></file:outbound-endpoint>
</foreach>
<set-property propertyName="Content-Type" value="application/json" doc:name="Content-Type" />
<set-payload value="{'result': 'ok'}" doc:name="Set Payload" />
</flow>

Some prerequisites before you begin. You’ll need the following:

• Amazon AWS Account with Access Key and Secret Key
• MuleSoft Anypoint Studio
• Amazon S3 Bucket

Let me highlight some of the key areas in the code.

• When you run the project, you’ll kick off the process by navigating to http://localhost:8081/bucket. This is set in the HTTP Listener.
• If you’ve pasted my code above into your project, you’ll need to configure the Amazon S3 configuration and the HTTP configuration
• For the S3 Connector, you’ll need your Amazon AWS Access Key and Secret Key when you set up the configuration. Be sure to click on Test Connection before hitting OK.

  

• The first Amazon S3 Connector grabs a list of all the Objects with in a Bucket. The field Bucket Name is hard-coded. You can change this to be dynamic and use a message.inboundProperties value instead.
• You’ll need to open the properties for the File Connector and set the Path to a local directory before running the project.

With this example, you can see how easy it is to integrate with Amazon S3 without having to write any custom code. The connector provides a wrapper around the Amazon SDK to allow you the ability easily click-and-configure a project.