[Esip-cloud] Getting data into the cloud with AWS CLI
rob at iris.washington.edu
Tue Aug 10 15:08:54 EDT 2021
I am an AWS CLI newb, but thought I would ask some questions to get clarification on the issue. It sounds like you ultimately were able to get your content loaded, but in a very roundabout way. Your aim was to be able to use the 'cp' command directly, as I understand it.
> I am trying to copy large file to s3 using these commands:
> aws s3 cp LEN-DB.hdf5 s3://esip-qhub/unavco/globalEarthquakes/ <s3://esip-qhub/unavco/globalEarthquakes/>
> aws s3 cp LEN-DB.hdf5 s3://esip-qhub/unavco/globalEarthquakes <s3://esip-qhub/unavco/globalEarthquakes>
> aws s3 cp LEN-DB.hdf5 s3://esip-qhub/unavco <s3://esip-qhub/unavco>
> aws s3 cp LEN-DB.hdf5 s3://esip-qhub/globalEarthquakes <s3://esip-qhub/globalEarthquakes>
> All of these fail with:
> upload failed: ./LEN-DB.hdf5 to s3://esip-qhub/unavco <s3://esip-qhub/unavco> An error occurred (AccessDenied) when calling the CreateMultipartUpload operation: Access Denied
I am curious if you got this same message on the very last command you entered, given the fact that the /unavco bucket wasn't even invoked. This at first glance looks like bucket-level access control.
> I believe that the instructions in reading_writing_data.ipynb may only work if you have superuser permissions… or if you have aws cli installed. If neither of these are true, when you try to run: asw configure —profile esip-qhub you get the
> bash aws not found error.
I noticed in your command example that you misspelled 'aws', was that the cause of your error or was an an email typo?
> Then this command works.
> /home/jovyan/.local/bin/aws configure --profile esip-qhub
It seems that ultimately, you were able to get the configure for your credentials to work by an explicit path to 'aws'. Does the base OS image not have a default path to 'aws'? Does the 'which' command work?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Esip-cloud