Bulk Restore from S3 Glacier

I'll be using S3 compatible storage with Scaleway however this is the same process for Amazon AWS and other object storage providers.

Ideally this would be nice if it was baked into the Web UI however with Scaleway this is limited to one object at a time and isn't viable if you have thousands of objects like me.

We'll be using the s3cmd tool to quickly restore all our objects, this is an open source tool and the code can be found on GitHub https://github.com/s3tools/s3cmd.

Please note you will likely incur restore costs.

Install s3cmd
sudo apt install s3cmd

Create config file:

cd $HOME
touch .s3cfg

Enter the file and use fill in the template:

[default]
host_base = s3.nl-ams.scw.cloud
host_bucket = BUCKET-HERE.s3.nl-ams.scw.cloud
bucket_location = nl-ams
use_https = True
# Login credentials
access_key = <ACCESS_KEY>
secret_key = <SECRET_KEY>

Calculate the number of files in Glacier that are to be restored:
s3cmd ls -l --recursive s3://BUCKET-HERE |grep GLACIER|wc -l

Restore for 3 days
s3cmd restore -v --recursive s3://<bucket-name> --restore-days=3

Permanent Restore
Due to the way s3cmd and the S3 API works there is no way to indefinitely restore the objects. The easy way would be to set the restore days to the max of 10000 or 27 years.

However to do it properly, after temporarily restoring the files we can move them which will remove the restore duration tied to them.

s3cmd mv s3://your-bucket-name/path/to/object s3://your-bucket-name/path/to/destination --recursive