We needed to move the uploaded images from Digital Ocean Droplets to Digital Ocean Spaces. But there was over 100GB to transfer.
Spaces is an S3 object storage service from DigitalOcean. In simple terms, it’s hosting for files, and we’ll be using it to store the WordPress uploads, which is everything you’ll find in the Media section of a WordPress site.
To connect to DigitalOcean Spaces, you’ll need an access key and secret key. You can find these on the “Applications & API” page of the DigitalOcean control panel. They will be needed during the setup for rclone.
Going through the whole process of creating a new remote by running
rclone config, each prompt should be answered as shown in the steps below:
Creating a new rlcone config for transferring files to S3
- SSH into your server
You’ll need to use the root user
- Install rclone if you haven’t already
$ curl https://rclone.org/install.sh | sudo bash
- Open rlcone
$ rclone config
- Create a new config
- Give the config a name
- Choose your storage type
- Choose your S3 provider
- Use AWS credentials
This is false, as we’re using Digital Ocean, not AWS
- Enter your access key
- Enter your secret key
- Choose your region
In this case, it’s null so just hit return, you don’t need to type anything
- Add your endpoint
The first time I ran this, I simply put in the endpoint, starting at nyc1, and that actually created a new Digital Ocean Space when I ran it. This could be useful in some cases, but for our purposes, we only wanted to add to an existing space, not create a new one each time we run the transfer.
- Set location contraints
- Set Access Rights
Leave as default if you want all the files to be private. In our case, we wanted anyone with the URL to be able to access a file. Simply set this to “2” for ‘public-read’
- Edit Advanced Config
For our purposes, we skipped this.
- Save your config
Once that’s all done, you can close out of rclone config by typing “q” and hitting return.
To run rlcone with your newly created config, run the following:
rclone copy /path/to/folder yourConfigName:siteFolder