Start a Project

Using rclone to move WordPress uploads to Digital Ocean Spaces

Karl – a web developer with Wibble web design Belfast
By Karl Proctor 29 November, 20195 MIN READ
Wibble Blog - Using rclone to move WordPress uploads to Digital Ocean Spaces

We needed to move the uploaded images from Digital Ocean Droplets to Digital Ocean Spaces. But there was over 100GB to transfer.

Spaces is an S3 object storage service from DigitalOcean. In simple terms, it’s hosting for files, and we’ll be using it to store the WordPress uploads, which is everything you’ll find in the Media section of a WordPress site.

To connect to DigitalOcean Spaces, you’ll need an access key and secret key. You can find these on the “Applications & API” page of the DigitalOcean control panel. They will be needed during the setup for rclone.

Going through the whole process of creating a new remote by running rclone config, each prompt should be answered as shown in the steps below:

Time needed: 45 minutes

Creating a new rlcone config for transferring files to S3

  1. SSH into your server

    You’ll need to use the root user

  2. Install rclone if you haven’t already

    $ curl https://rclone.org/install.sh | sudo bash

  3. Open rlcone

    $ rclone config

  4. Create a new config

    > n

  5. Give the config a name

    name> exampleS3

  6. Choose your storage type

    storage> s3

  7. Choose your S3 provider

    provider> DigitalOcean

  8. Use AWS credentials

    This is false, as we’re using Digital Ocean, not AWS
    env_auth> 1
    or
    env_auth> false

  9. Enter your access key

    access_key_id> yourAccessKey

  10. Enter your secret key

    secret_access_key> yourSecretKey

  11. Choose your region

    In this case, it’s null so just hit return, you don’t need to type anything
    region>

  12. Add your endpoint

    endpoint> yourSpaceName.nyc1.digitaloceanspaces.com

    The first time I ran this, I simply put in the endpoint, starting at nyc1, and that actually created a new Digital Ocean Space when I ran it. This could be useful in some cases, but for our purposes, we only wanted to add to an existing space, not create a new one each time we run the transfer.

  13. Set location contraints

    location_constraint>

  14. Set Access Rights

    Leave as default if you want all the files to be private. In our case, we wanted anyone with the URL to be able to access a file. Simply set this to “2” for ‘public-read’
    acl> 2
    or
    acl> public-read

  15. Edit Advanced Config

    For our purposes, we skipped this.
    y/n> n

  16. Save your config

    y/e/d> y

Once that’s all done, you can close out of rclone config by typing “q” and hitting return.

To run rlcone with your newly created config, run the following:

rclone copy /path/to/folder yourConfigName:siteFolder

Conclusion

This post is the second part of our two part series on using Digital Ocean to manage images in the wp-content/uploads folder. Part one is here: Using Digital Ocean Spaces on a WordPress site.

This development is another part of Wibble’s managed web hosting offering – it is all about Wibble instigating features to improve our service to our amazing web design clients. We introduced this feature to help with site speed, server size and, potentially, disaster recovery.

We are delighted with the effect it has had on our automated backups and we plan to role out more similar features to help improve the offering to all our clients.


Share this blog post

Karl – a web developer with Wibble web design Belfast

Karl Proctor

More from author