SQS (https://aws.amazon.com/sqs/) is a subset of what you get with RabbitMQ, but it is often good enough for most use cases.
You can create two types of queue. A First-In-First-Out (FIFO) stores messages in the order they are received, and ensures that a message that's retrieved from the queue is read just once. They are useful when you want to store a stream of messages that need to be picked up by workers, like what you would do with Celery and Redis. They have a limit of 20,000 in-flight messages.
The second type (standard) is similar, except that the ordering is not entirely guaranteed. That makes it much faster than the FIFOs, and has a higher limit (120,000).
The messages stored in SQS are replicated in several AZs in the AWS cloud, making them reliable.
AWS is organized into Regions and in each Region, Availability Zones.
Regions are isolated one from each other to ensure fault tolerance and stability. AZ are also isolated but they are attached with low-latency links. Instances spread across different AZ in the same region can be used behind the same load balancer in AWS.
Since the maximum size of a message is 256 KB, the volume you can store in a FIFO queue is 5 GB, and it is 30 GB for the standard one. In other words, there are no real limitations besides the price.