If you would like to sync a Windows folder to S3 storage, you can do it in just a few easy steps. That way, you can have your files backed up in the cloud and still have them accessible in the cloud console. You can also sync your files across multiple machines.

There are two ways to sync a local Windows folder to ORock S3 storage. This blog will show you a simple way to do it using Windows task scheduler and another way using Jenkins (which has a plethora of plug-ins) to sync on a non-cadence schedule, such as when a folder’s content changes.

Using Windows ‘task scheduler’

Tools needed:

Here are the tools that you will need:

• Rclone for Windows: https://downloads.rclone.org/v1.56.0/rclone-v1.56.0-windows-amd64.zip
• Note pad: Included in Windows
• cmd: included in Windows

Setup rclone:

To get started, we need to setup rclone to work with our S3 storage in ORock.

1. Download rclone from the link above.
2. Create a directory to work in. I am going to use c:\rclone. Create this using File Explorer.
3. Extract rclone.exe from the zip file and copy rclone.exe to c:\rclone
4. From the windows task bar type ‘cmd’ and open ‘Command prompt’
5. Change to the c:\rlcone directory: cd c:\rclone
6. Setup rclone to talk to ORock S3 storage.
   • Get the API access which are found on the ORock console API access page: https://console.orocktech.com/project/api_access/
   • Click on ‘View Credentials’. You will need the following:
     1. S3 URL
     2. EC2 Access Key
     3. EC2 Secret Key. To see the key, click on the eye icon in the box to the right.
   • Start rclone configuration. We will take the defaults for many of the options.
     1. 1. 1. $ rclone.exe config (enter)
           2. ‘n’ (enter) <- “new remote”
           3. name>orock (enter)
           4. Storage>4 (enter) <- “Amazon S3 Compliant Storage…”
           5. Provider>14 (enter) <- “Any other S3 compatible provider”
           6. env_auth>(enter) <- taking the default option
           7. access_key_id><EC2 Access Key from credentials> (enter)
           8. secret_access_key><EC2 Secret Key from credentials> (enter)
           9. region> (enter) <- taking the default
           10. endpoint> <S3 URL from credentials> (enter)
           11. location_constraint> (enter) <- taking default
           12. acl> (enter) <- taking the default
           13. Edit advanced config? y/n> (enter) <-taking the default
           14. It will show what you have supplied. y/e/d> (enter)  <-taking the default
           15. e/n/d/r/c/w/q>q (enter) <- we are done with configuration

7. Test if everything is working well.
   • $ rclone lsd orock:
     1. You should see a list of buckets that you have.
8. Create a bucket if one does not exist.
   1. 1. • $ rclone mkdir orock:mybucket

Create a directory to sync to:

Create a directory to sync to. I am going to use c:\rclone\orock. Create this using File Explorer.

Create a batch file that will do the sync operations.

I’m going to use the rclone sub command of copy and sync.  Copy all the local files to the S3 storage.  Sync will sync the S3 storage with the local folder, deleting any local files that are not in the S3 storage.  There are many options for rclone.  See rclone documentation for options.

1. Create a text file in c:\rclone\ named sync.txt. Can be done with file explorer.
2. Edit sync.txt. Add the following lines:

• • c:\rclone\rclone.exe copy -P c:\rclone\orock orock:mybucket
  • c:\rclone\rclone.exe sync -P orock:mybucket c:\rclone\orock –delete-after

3. Save the file.
4. Change the file to be a batch file
   • $ rename sync.txt sync.bat
5. Run the batch file to make sure everything is working. You should see information on the connection and throughput.
   • $ sync.bat

Create a Windows task to run the sync every 5 minutes using ‘task scheduler’

1. From the windows task bar type ‘task scheduler’ and open the windows ‘task scheduler’
2. Click on ‘Task Scheduler Library’
3. On the right side click on ‘Create Task…’
4. Give the task a name ‘s3sync’
5. Click on ‘Run whether user is logged on or not’ and ‘Do not store password……’ This will allow the bath file to run in the background.
6. Click on ‘Hidden’. This is to hide the task and run it in the background.
7. Click on the ‘Triggers’ tab.
8. Click on ‘New…’
9. Leave the settings the default. It is ‘One time’, which is a little bit confusing, but is what is needed.
10. Click on ‘repeat task every:’
11. Change the drop down to ‘5 minutes’.
12. Change the ‘for a duration of:’ drop down to ‘Indefinitely’
13. Click the ‘OK’ button.
14. Click the ‘Actions’ tab.
15. Click the “New…” button.
16. Click on the ‘Browse…’ button.
17. Navigate to c:\rclone\ and select rsync.bat.
18. Click ‘Open’
19. Click ‘OK’ button. Other tabs can be used to tweak the task as you like.
20. Click ‘OK’ button to create the task

At this time, you should see the task in the tasks Windows.  You can right click on the task and run it.  It should show the next run time.  You will need to click on the refresh button to get accurate information on the schedules.

Everything is setup and your task should now run every 5 minutes, syncing to S3 storage.

Using Jenkins on Windows

Tools needed:

• Rclone for Windows: https://downloads.rclone.org/v1.56.0/rclone-v1.56.0-windows-amd64.zip
• Java JRE: https://javadl.oracle.com/webapps/download/AutoDL?BundleId=245057_d3c52aa6bfa54d3ca74e617f18309292
• Jenkins: https://www.jenkins.io/download/thank-you-downloading-windows-installer-stable

Setup rclone

Use the instructions above.

Install Java

Run the java JRE installer downloaded from above. Use default options.

Allow user to logon as a service

For Jenkins to run correctly, we need to allow the user to log on as a service.

1. From the windows task bar type ‘local security policy’ and open the windows ‘local security policy’
2. Click on ‘Security Settings’->’Local Policies’->’User Rights Assignments’
3. Double click ‘Log on as a service’.
4. Click button ‘Add User or Group…’
5. Add your logon to the ‘Enter the object names…’ box.
6. Click ‘OK’ button.
7. Reboot for the security changes to take effect.

Install Jenkins

Now we want to install and configure Jenkins.  We will use the defaults for everything.

1. Run the Jenkins installer downloaded from the link above.
2. For ‘Service Long Credentials’ enter your logon account and password. Click ‘Test Credentials’ to make sure everything is working.
3. For ‘Port Selection’ you will also need to click on ‘Test Port’.
4. For ‘Select Java Home directory’ change it to C:\Program Files (x86)\Java\jre1.8.0_301 where we installed Java.

Jenkins should be installed and take you to the initial setup screen.

Setup Jenkins

You should be in an internet browser now.

1. Follow the setup instructions and use the ‘Install suggested plugins’.
2. Create an Admin user.
3. Change the URL to a FQDN if you want to access Jenkins from another machine. You will also need to open the port in the firewall, which is done during install.

Setup the Jenkins job to run

You should still be in an internet browser.  We want to create our job to run the sync.

1. Click on ‘Create a job’
2. Give the job a name. I am going to use s3sync.
3. Click on ‘Freestyle project.
4. Click on ‘OK’ button.

There are many options to choose from.  For this job, we are going to keep it simple.  You can click on the ‘?’ to see what many of the options do.  You can find many different plugins if you want to add to the options.

5. Check ‘Build periodically’.
6. Add ‘*/5 * * * *’ to the box that just appeared.

Next, we want to add a build step where we will run our sync.

7. Click ‘Add build step’->’Execute Windows batch command’
8. Add the following lines to the command windows.

• • c:\rclone\rclone.exe copy -P c:\rclone\orock orock:mybucket
  • c:\rclone\rclone.exe sync -P orock:mybucket c:\rclone\orock –delete-after

9. Click ‘Save’

Currently, you have setup sync to S3.  In Jenkins, you can view the log files that it creates.  You can also add other triggers or build actions.  There are many options available and many plug-ins you can add.