Skip to main content

Backups failing to S3 with 400 Bad Request error

slim
All my servers seem to exhibit similar problems when transfering the backups to S3. Some accounts make it, so the bucket is setup correctly, and I can validate the settings in WHM. When the backups attempt to transfer the files to S3 I get this in the logs: [2015-05-08 08:32:05 +1000] warn [cpbackup_transporter] Upload attempt failed: Amazon::S3: Amazon responded with 400 Bad Request cPanel priority support pointed me to the AWS error responses page, which is not helpful. Has anyone else had this issue? Currently my s3 backups contain only a few accounts - no where near all of them

Comments

13 comments

Date Votes
  • cPanelMichael
    Hello, This is discussed (with a solution on the last post) on the following thread:
    0
  • slim
    Hi Unfortunately the size of the backup doesn't seem to be the issue. Many of my sites are < 5gig.
    0
  • cPanelPeter cPanel Staff
    Hello, You mentioned cPanel priority support? What was the ticket number?
    0
  • slim
    Hi - ticket # 6473249 - Not that the support guy seems to have done anything bad.. The error is so generic that AWS documentation doesn't narrow it down to anything - And AWS provide zero support unless your on a contract. I figure thou that if the verification of the S3 bucket passes, it means I have done everything correct - as in the bucket setup and permissions are correct. There isnt much else I can do from AWS side to make it work. I just checked all my backup buckets - Very little has been uploaded by cpanel backup in may - only a few accounts have made it over to S3. prior to that it seemed ok. Maybe it was an update cPanel have done?
    0
  • cPanelMichael
    Hello, Per the support ticket, the issue is that Amazon is rejecting the transfer with the "400 - Bad Request" error message. This indicates the issue is with Amazon's server. Are you sure there is no support available from Amazon? I do see they offer support forums at:
    0
  • slim
    Yep, they do - And this is the result:
    0
  • cPanelMichael
    Hello, Have you tried enabling logging on the Amazon server to see if more details are available about the rejection? Thank you.
    0
  • slim
    I enabled logging - it creates several log files.. Here is an example:
    MYBUCKETNAMEHERE [15/May/2015:00:41:26 +0000] 10.150.10.176 6416644E32CEB557 REST.GET.BUCKET - "GET /MYBUCKETNAMEHERE?prefix=&max-keys=100&marker=&delimiter=/&encoding-type=url HTTP/1.1" 200 - 420 - 10 10 "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:41:26 +0000] 10.150.10.176 949420431A6B89B2 REST.GET.VERSIONING - "GET /MYBUCKETNAMEHERE?versioning HTTP/1.1" 200 - 113 - 18 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:41:27 +0000] 10.150.10.176 7B4E1E68FD5F3BEF REST.GET.REQUEST_PAYMENT - "GET /MYBUCKETNAMEHERE?requestPayment HTTP/1.1" 200 - 172 - 3 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:41:27 +0000] 10.150.10.176 92BF6606CF57FEB1 REST.GET.VERSIONING - "GET /MYBUCKETNAMEHERE?versioning HTTP/1.1" 200 - 113 - 17 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:41:27 +0000] 10.150.10.176 D9FFA442A1577FB4 REST.GET.TAGGING - "GET /MYBUCKETNAMEHERE?tagging HTTP/1.1" 404 NoSuchTagSet 289 - 43 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:41:31 +0000] 10.150.10.176 E9B60FF5B4E24A9B REST.GET.BUCKET - "GET /MYBUCKETNAMEHERE?prefix=&max-keys=100&marker=&delimiter=/&encoding-type=url HTTP/1.1" 200 - 420 - 13 12 "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:43:09 +0000] 10.150.10.176 3332B396431E94E0 REST.GET.BUCKET - "GET /MYBUCKETNAMEHERE?prefix=weekly/&max-keys=100&marker=weekly/&delimiter=/&encoding-type=url HTTP/1.1" 200 - 452 - 13 12 "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:44:44 +0000] 10.150.10.176 0F3B898981E00FFE REST.GET.REPLICATION - "GET /?replication HTTP/1.1" 404 ReplicationConfigurationNotFoundError 332 - 6 - "-" "aws-internal/3" - MYBUCKETNAMEHERE [15/May/2015:00:44:45 +0000] 10.150.10.176 0E80B50B1C4197EF REST.GET.LIFECYCLE - "GET /MYBUCKETNAMEHERE?lifecycle HTTP/1.1" 404 NoSuchLifecycleConfiguration 322 - 16 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:44:45 +0000] 10.150.10.176 4BC8CD29D314A649 REST.GET.CORS - "GET /MYBUCKETNAMEHERE?cors HTTP/1.1" 404 NoSuchCORSConfiguration 312 - 14 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:39 +0000] 10.150.10.176 5C2875D747848AB9 REST.GET.TAGGING - "GET /MYBUCKETNAMEHERE?tagging HTTP/1.1" 404 NoSuchTagSet 289 - 67 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:39 +0000] 10.150.10.176 F9E3DD11F717E76B REST.GET.NOTIFICATION - "GET /?notification HTTP/1.1" 200 - 115 - 19 - "-" "aws-internal/3" - MYBUCKETNAMEHERE [15/May/2015:00:45:42 +0000] 10.150.10.176 269789AE437BB474 REST.GET.NOTIFICATION - "GET /?notification HTTP/1.1" 200 - 115 - 14 - "-" "aws-internal/3" - MYBUCKETNAMEHERE [15/May/2015:00:45:43 +0000] 10.150.10.176 09A95372E7068DE7 REST.GET.CORS - "GET /MYBUCKETNAMEHERE?cors HTTP/1.1" 404 NoSuchCORSConfiguration 312 - 2 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:43 +0000] 10.150.10.176 6F747B38AA7B372B REST.GET.LIFECYCLE - "GET /MYBUCKETNAMEHERE?lifecycle HTTP/1.1" 404 NoSuchLifecycleConfiguration 322 - 2 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:44 +0000] 10.150.10.176 30051FF61E9AD8AD REST.GET.NOTIFICATION - "GET /?notification HTTP/1.1" 200 - 115 - 16 - "-" "aws-internal/3" - MYBUCKETNAMEHERE [15/May/2015:00:45:45 +0000] 10.150.10.176 20D02AFA7F5008DD REST.GET.BUCKET - "GET /MYBUCKETNAMEHERE?prefix=weekly/&max-keys=100&marker=weekly/&delimiter=/&encoding-type=url HTTP/1.1" 200 - 452 - 10 10 "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:45 +0000] 10.150.10.176 AB3CB1F7CC868E85 REST.GET.BUCKETPOLICY - "GET /MYBUCKETNAMEHERE?policy HTTP/1.1" 404 NoSuchBucketPolicy 302 - 19 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:45 +0000] 10.150.10.176 C113F27784D1ED9C REST.GET.WEBSITE - "GET /MYBUCKETNAMEHERE?website HTTP/1.1" 404 NoSuchWebsiteConfiguration 336 - 11 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:45 +0000] 10.150.10.176 F11849534B35783D REST.GET.ACL - "GET /MYBUCKETNAMEHERE?acl HTTP/1.1" 200 - 947 - 1072 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:48 +0000] 10.150.10.176 498A3FD9505D5DBF REST.GET.TAGGING - "GET /MYBUCKETNAMEHERE?tagging HTTP/1.1" 404 NoSuchTagSet 289 - 42 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:48 +0000] 10.150.10.176 DF8EFE2B198968F7 REST.GET.LOGGING_STATUS - "GET /MYBUCKETNAMEHERE?logging HTTP/1.1" 200 - 247 - 18 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:48 +0000] 10.150.10.176 E8D2CAF9986870EB REST.GET.VERSIONING - "GET /MYBUCKETNAMEHERE?versioning HTTP/1.1" 200 - 113 - 2 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:49 +0000] 10.150.10.176 2D12A5F202975FA0 REST.GET.LOGGING_STATUS - "GET /MYBUCKETNAMEHERE?logging HTTP/1.1" 200 - 247 - 4 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:49 +0000] 10.150.10.176 6C0D8A3396CA2067 REST.GET.VERSIONING - "GET /MYBUCKETNAMEHERE?versioning HTTP/1.1" 200 - 113 - 4 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:50 +0000] 10.150.10.176 0DA0872E2A257966 REST.GET.REPLICATION - "GET /?replication HTTP/1.1" 404 ReplicationConfigurationNotFoundError 332 - 6 - "-" "aws-internal/3" - MYBUCKETNAMEHERE [15/May/2015:00:45:50 +0000] 10.150.10.176 69FB2819D4B2F507 REST.GET.BUCKETPOLICY - "GET /MYBUCKETNAMEHERE?policy HTTP/1.1" 404 NoSuchBucketPolicy 302 - 7 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:51 +0000] 10.150.10.176 8AC522589B2DF295 REST.GET.ACL - "GET /MYBUCKETNAMEHERE?acl HTTP/1.1" 200 - 947 - 1033 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:51 +0000] 10.150.10.176 A3EA18868E2E556C REST.GET.BUCKETPOLICY - "GET /MYBUCKETNAMEHERE?policy HTTP/1.1" 404 NoSuchBucketPolicy 302 - 8 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:51 +0000] 10.150.10.176 CF4F19697FC8D538 REST.GET.TAGGING - "GET /MYBUCKETNAMEHERE?tagging HTTP/1.1" 404 NoSuchTagSet 289 - 33 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:49:17 +0000] 10.150.10.176 81BBAA0C781CB766 REST.GET.VERSIONING - "GET /MYBUCKETNAMEHERE?versioning HTTP/1.1" 200 - 113 - 16 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:49:17 +0000] 10.150.10.176 9DE7577A391BBF1B REST.GET.BUCKET - "GET /MYBUCKETNAMEHERE?prefix=&max-keys=100&marker=&delimiter=/&encoding-type=url HTTP/1.1" 200 - 420 - 10 10 "-" "S3Console/0.4" -
    and
    MYBUCKETNAMEHERE [15/May/2015:00:41:26 +0000] 10.150.10.176 6416644E32CEB557 REST.GET.BUCKET - "GET /MYBUCKETNAMEHERE?prefix=&max-keys=100&marker=&delimiter=/&encoding-type=url HTTP/1.1" 200 - 420 - 10 10 "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:41:26 +0000] 10.150.10.176 949420431A6B89B2 REST.GET.VERSIONING - "GET /MYBUCKETNAMEHERE?versioning HTTP/1.1" 200 - 113 - 18 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:41:27 +0000] 10.150.10.176 7B4E1E68FD5F3BEF REST.GET.REQUEST_PAYMENT - "GET /MYBUCKETNAMEHERE?requestPayment HTTP/1.1" 200 - 172 - 3 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:41:27 +0000] 10.150.10.176 92BF6606CF57FEB1 REST.GET.VERSIONING - "GET /MYBUCKETNAMEHERE?versioning HTTP/1.1" 200 - 113 - 17 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:41:27 +0000] 10.150.10.176 D9FFA442A1577FB4 REST.GET.TAGGING - "GET /MYBUCKETNAMEHERE?tagging HTTP/1.1" 404 NoSuchTagSet 289 - 43 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:41:31 +0000] 10.150.10.176 E9B60FF5B4E24A9B REST.GET.BUCKET - "GET /MYBUCKETNAMEHERE?prefix=&max-keys=100&marker=&delimiter=/&encoding-type=url HTTP/1.1" 200 - 420 - 13 12 "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:43:09 +0000] 10.150.10.176 3332B396431E94E0 REST.GET.BUCKET - "GET /MYBUCKETNAMEHERE?prefix=weekly/&max-keys=100&marker=weekly/&delimiter=/&encoding-type=url HTTP/1.1" 200 - 452 - 13 12 "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:44:44 +0000] 10.150.10.176 0F3B898981E00FFE REST.GET.REPLICATION - "GET /?replication HTTP/1.1" 404 ReplicationConfigurationNotFoundError 332 - 6 - "-" "aws-internal/3" - MYBUCKETNAMEHERE [15/May/2015:00:44:45 +0000] 10.150.10.176 0E80B50B1C4197EF REST.GET.LIFECYCLE - "GET /MYBUCKETNAMEHERE?lifecycle HTTP/1.1" 404 NoSuchLifecycleConfiguration 322 - 16 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:44:45 +0000] 10.150.10.176 4BC8CD29D314A649 REST.GET.CORS - "GET /MYBUCKETNAMEHERE?cors HTTP/1.1" 404 NoSuchCORSConfiguration 312 - 14 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:39 +0000] 10.150.10.176 5C2875D747848AB9 REST.GET.TAGGING - "GET /MYBUCKETNAMEHERE?tagging HTTP/1.1" 404 NoSuchTagSet 289 - 67 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:39 +0000] 10.150.10.176 F9E3DD11F717E76B REST.GET.NOTIFICATION - "GET /?notification HTTP/1.1" 200 - 115 - 19 - "-" "aws-internal/3" - MYBUCKETNAMEHERE [15/May/2015:00:45:42 +0000] 10.150.10.176 269789AE437BB474 REST.GET.NOTIFICATION - "GET /?notification HTTP/1.1" 200 - 115 - 14 - "-" "aws-internal/3" - MYBUCKETNAMEHERE [15/May/2015:00:45:43 +0000] 10.150.10.176 09A95372E7068DE7 REST.GET.CORS - "GET /MYBUCKETNAMEHERE?cors HTTP/1.1" 404 NoSuchCORSConfiguration 312 - 2 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:43 +0000] 10.150.10.176 6F747B38AA7B372B REST.GET.LIFECYCLE - "GET /MYBUCKETNAMEHERE?lifecycle HTTP/1.1" 404 NoSuchLifecycleConfiguration 322 - 2 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:44 +0000] 10.150.10.176 30051FF61E9AD8AD REST.GET.NOTIFICATION - "GET /?notification HTTP/1.1" 200 - 115 - 16 - "-" "aws-internal/3" - MYBUCKETNAMEHERE [15/May/2015:00:45:45 +0000] 10.150.10.176 20D02AFA7F5008DD REST.GET.BUCKET - "GET /MYBUCKETNAMEHERE?prefix=weekly/&max-keys=100&marker=weekly/&delimiter=/&encoding-type=url HTTP/1.1" 200 - 452 - 10 10 "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:45 +0000] 10.150.10.176 AB3CB1F7CC868E85 REST.GET.BUCKETPOLICY - "GET /MYBUCKETNAMEHERE?policy HTTP/1.1" 404 NoSuchBucketPolicy 302 - 19 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:45 +0000] 10.150.10.176 C113F27784D1ED9C REST.GET.WEBSITE - "GET /MYBUCKETNAMEHERE?website HTTP/1.1" 404 NoSuchWebsiteConfiguration 336 - 11 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:45 +0000] 10.150.10.176 F11849534B35783D REST.GET.ACL - "GET /MYBUCKETNAMEHERE?acl HTTP/1.1" 200 - 947 - 1072 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:48 +0000] 10.150.10.176 498A3FD9505D5DBF REST.GET.TAGGING - "GET /MYBUCKETNAMEHERE?tagging HTTP/1.1" 404 NoSuchTagSet 289 - 42 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:48 +0000] 10.150.10.176 DF8EFE2B198968F7 REST.GET.LOGGING_STATUS - "GET /MYBUCKETNAMEHERE?logging HTTP/1.1" 200 - 247 - 18 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:48 +0000] 10.150.10.176 E8D2CAF9986870EB REST.GET.VERSIONING - "GET /MYBUCKETNAMEHERE?versioning HTTP/1.1" 200 - 113 - 2 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:49 +0000] 10.150.10.176 2D12A5F202975FA0 REST.GET.LOGGING_STATUS - "GET /MYBUCKETNAMEHERE?logging HTTP/1.1" 200 - 247 - 4 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:49 +0000] 10.150.10.176 6C0D8A3396CA2067 REST.GET.VERSIONING - "GET /MYBUCKETNAMEHERE?versioning HTTP/1.1" 200 - 113 - 4 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:50 +0000] 10.150.10.176 0DA0872E2A257966 REST.GET.REPLICATION - "GET /?replication HTTP/1.1" 404 ReplicationConfigurationNotFoundError 332 - 6 - "-" "aws-internal/3" - MYBUCKETNAMEHERE [15/May/2015:00:45:50 +0000] 10.150.10.176 69FB2819D4B2F507 REST.GET.BUCKETPOLICY - "GET /MYBUCKETNAMEHERE?policy HTTP/1.1" 404 NoSuchBucketPolicy 302 - 7 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:51 +0000] 10.150.10.176 8AC522589B2DF295 REST.GET.ACL - "GET /MYBUCKETNAMEHERE?acl HTTP/1.1" 200 - 947 - 1033 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:51 +0000] 10.150.10.176 A3EA18868E2E556C REST.GET.BUCKETPOLICY - "GET /MYBUCKETNAMEHERE?policy HTTP/1.1" 404 NoSuchBucketPolicy 302 - 8 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:45:51 +0000] 10.150.10.176 CF4F19697FC8D538 REST.GET.TAGGING - "GET /MYBUCKETNAMEHERE?tagging HTTP/1.1" 404 NoSuchTagSet 289 - 33 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:49:17 +0000] 10.150.10.176 81BBAA0C781CB766 REST.GET.VERSIONING - "GET /MYBUCKETNAMEHERE?versioning HTTP/1.1" 200 - 113 - 16 - "-" "S3Console/0.4" - MYBUCKETNAMEHERE [15/May/2015:00:49:17 +0000] 10.150.10.176 9DE7577A391BBF1B REST.GET.BUCKET - "GET /MYBUCKETNAMEHERE?prefix=&max-keys=100&marker=&delimiter=/&encoding-type=url HTTP/1.1" 200 - 420 - 10 10 "-" "S3Console/0.4" -
    0
  • cPanelMichael
    Hello, To clarify, were those log entries populated when the backup generation failed? Also, please ensure you add long snippets of log entries in "CODE" tags when posting on the forums. Thank you.
    0
  • slim
    I'll check back tomorrow - after the backups run.
    0
  • cPanelMichael
    I'll check back tomorrow - after the backups run.

    You can force a backup generation via:
    /usr/local/cpanel/bin/backup --force
    Thank you.
    0
  • slim
    Found some stuff in logs that looks related
    MYBUCKET [17/May/2015:23:03:46 +0000] 175.107.161.239 arn:aws:iam::274956579896:user/USERNAME C433850517EE7051 REST.PUT.PART 2015-05-18/MYBACKUPFILE.tar.gz "PUT /2015-05-18%MYBACKUPFILE%2Etar%2Egz?partNumber=1&uploadId=pWGCcuvCwZ6XrnTaJWEvG71uhu9E2eNeHvXYm.yOfs1tOqcjcTJX.Z4DQCAaFMwi7bm4elIFsVtpGhyNcFpaYVIavIGwe4AKCWhWebH7NWSl_MPD9LhAj4O6ZW2XMIvx5lH6Ragl1UE81iEPXSi8kQ-- HTTP/1.1" 400 RequestTimeout 360 20971520 20036 - "-" "libwww-perl/6.05" -
    That backup file is about 30meg - and didn't make it to the S3 server.
    0
  • cPanelMichael
    Hello, That error simply shows: "400 Request Timeout" You will need to report this issue to Amazon to let them know your service is not working as expected. It's outside the control of the cPanel server to resolve a timeout issue with the Amazon service. Thank you.
    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post