AWS: Here's how you subscribe an SQS queue to an SNS topic!

Image via Wikipedia

There's precious few code examples for subscribing an Amazon Web Services' SQS queue to an SNS topic, especially using the SDK for Java. I've finally figured it out, and this is my way of giving back to the authors of all the hints I could find around the Web. The tricky part is the queue policy; if you create it incorrectly, you won't see any error messages -- it just won't work.

The methodology is this:

1. Create an SNS topic.
2. Set a policy on the topic to allow subscriptions.
3. Create an SQS queue.
4. Set a policy on the queue to allow message posting.
5. Subscribe the queue to the topic.
6. Wait for AWS settings to propagate and "take."
7. Send an SNS notification message.
8. Listen for the message on the queue.

And here's the code:


public class SnsWithSqsDemo {
private static final Log LOG = LogFactory.getLog(SnsWithSqsDemo.class);
private static final String TOPIC_NAME = "TestTopic";
private static final String QUEUE_NAME = "TestQueue";

/**
* This program demonstrates how to hook up an AWS SQS queue to an AWS SNS
* topic to receive messages.
*
* 1. Create an SNS topic.
* 2. Set a policy on the topic to allow subscriptions.
* 3. Create an SQS queue.
* 4. Set a policy on the queue to allow message posting.
* 5. Subscribe the queue to the topic.
* 6. Wait for AWS settings to propagate and "take."
* 7. Send an SNS notification message.
* 8. Listen for the message on the queue.
*
* @param args not used
*/
public static void main(String[] args) {
AWSCredentials awsCredentials = null;
AmazonSNS sns = null;
AmazonSQS sqs = null;
String topicArn = null;
String queueUrl = null;
String queueArn = null;
String subscriptionArn = null;
LOG.debug("Beginning.");
try {
awsCredentials =
new PropertiesCredentials(
SnsWithSqsDemo.class.getResourceAsStream("AwsCredentials.properties"));
sns = new AmazonSNSClient(awsCredentials);
// 1. Create a topic
System.out.println("Step 1. Create an SNS topic.");
CreateTopicResult ctResult =
sns.createTopic(new CreateTopicRequest(TOPIC_NAME));
topicArn = ctResult.getTopicArn();
System.out.println(String.format("Created topic %s with ARN %s",
TOPIC_NAME, topicArn));
// 2. Set policy on topic to allow open subscriptions
System.out.println("Step 2. Set a policy on the topic to allow subscriptions.");
Policy snsPolicy =
new Policy().withStatements(
new Statement(Effect.Allow)
.withPrincipals(Principal.AllUsers)
.withActions(SNSActions.Subscribe));
System.out.println("Set SNS policy: " + snsPolicy.toJson());
sns.setTopicAttributes(new SetTopicAttributesRequest(
topicArn, "Policy", snsPolicy.toJson()));
// 3. Create a queue
System.out.println("Step 3. Create an SQS queue.");
sqs = new AmazonSQSClient(awsCredentials);
CreateQueueResult cqResult =
sqs.createQueue(new CreateQueueRequest(QUEUE_NAME));
queueUrl = cqResult.getQueueUrl();
System.out.println(String.format("Created queue %s with URL %s",
QUEUE_NAME, queueUrl));
GetQueueAttributesResult queueArnResult =
sqs.getQueueAttributes(new GetQueueAttributesRequest(queueUrl)
.withAttributeNames("QueueArn"));
queueArn = queueArnResult.getAttributes().get("QueueArn");
System.out.println("Queue ARN = " + queueArn);
// 4. Set the queue policy to allow SNS to publish messages
System.out.println("Step 4. Set a policy on the queue to allow message posting.");
Policy sqsPolicy =
new Policy().withStatements(
new Statement(Effect.Allow)
.withPrincipals(Principal.AllUsers)
.withResources(new Resource(queueArn)) // Note: queue, not topic
.withActions(SQSActions.SendMessage)
.withConditions(
ConditionFactory.newSourceArnCondition(topicArn)));
Map queueAttributes = new HashMap();
queueAttributes.put("Policy", sqsPolicy.toJson());
sqs.setQueueAttributes(new SetQueueAttributesRequest(queueUrl, queueAttributes));
System.out.println("Set SQS policy to " + queueUrl + ": " + sqsPolicy.toJson());
// 5. Subscribe the queue to the topic
System.out.println("Step 5. Subscribe the queue to the topic.");
SubscribeResult sResult =
sns.subscribe(new SubscribeRequest(topicArn, "sqs", queueArn));
subscriptionArn = sResult.getSubscriptionArn();
System.out.println("Subscription ARN: " + subscriptionArn);
// 6. Wait a bit for AWS to get all synched-up
System.out.println("Step 6. Wait for AWS settings to propagate and \"take.\"");
Thread.sleep(60000L);
// 6.1. Verify queue attributes
GetQueueAttributesResult gqaResult =
sqs.getQueueAttributes(new GetQueueAttributesRequest(queueUrl)
.withAttributeNames("Policy", "QueueArn", "ApproximateNumberOfMessages"));
if (gqaResult.getAttributes().size() == 0) {
System.out.println("Queue " + QUEUE_NAME + " has no attributes");
} else {
System.out.println("Attributes for " + QUEUE_NAME);
for (String key : gqaResult.getAttributes().keySet()) {
System.out.println(String.format("\t%s = %s",
key, gqaResult.getAttributes().get(key)));
}
}
// 6.2. Verify topic attributes
GetTopicAttributesResult gtaResult =
sns.getTopicAttributes(new GetTopicAttributesRequest(topicArn));
if (gtaResult.getAttributes().size() == 0) {
System.out.println("Topic " + TOPIC_NAME + " has no attributes");
} else {
System.out.println("Attributes for " + TOPIC_NAME);
for (String key : gtaResult.getAttributes().keySet()) {
System.out.println(String.format("\t%s = %s",
key, gtaResult.getAttributes().get(key)));
}
}
// 6.3. Verify subscription
ListSubscriptionsByTopicResult lsbtResult =
sns.listSubscriptionsByTopic(new ListSubscriptionsByTopicRequest(topicArn));
if (lsbtResult.getSubscriptions().size() == 0) {
System.out.println("Topic " + TOPIC_NAME + " has no subscriptions.");
} else {
System.out.println("Subscriptions for " + TOPIC_NAME);
for (Subscription subscription : lsbtResult.getSubscriptions()) {
System.out.println("\t" + subscription.getProtocol() + ": "
+ subscription.getEndpoint());
}
}
// 7. Send a notification
System.out.println("Step 7. Send an SNS notification message.");
PublishResult pResult =
sns.publish(new PublishRequest(topicArn,
"Mr Watson -- Come here -- I want to see you."));
System.out.println("Sent message ID = " + pResult.getMessageId());
// 8. Wait for message receipt in queue
System.out.println("Step 8. Listen for the message on the queue.");
for (int i = 0; i < 10; i++) { Thread.sleep(2000L); ReceiveMessageResult rmResult = sqs.receiveMessage(new ReceiveMessageRequest(queueUrl)); if (rmResult.getMessages().size() > 0) {
// A message has been received
for (Message message : rmResult.getMessages()) {
System.out.println(message.getBody());
sqs.deleteMessage(new DeleteMessageRequest(queueUrl,
message.getReceiptHandle()));
}
break;
} else {
// ??? Why aren't we receiving messages?
System.out.println("No messages available, attempt " + (i+1));
}
}
} catch (Exception e) {
e.printStackTrace(System.err);
} finally {
System.out.println("Shutting down...");
// Unsubscribe the queue from the topic
if (sns != null && subscriptionArn != null) {
sns.unsubscribe(new UnsubscribeRequest(subscriptionArn));
System.out.println("Unsubscribed queue from topic.");
}
// Destroy queue
if (sqs != null && queueUrl != null) {
sqs.deleteQueue(new DeleteQueueRequest(queueUrl));
System.out.println("Deleted the queue.");
sqs.shutdown();
}
// Destroy topic
if (sns != null && topicArn != null) {
sns.deleteTopic(new DeleteTopicRequest(topicArn));
System.out.println("Deleted the topic.");
sns.shutdown();
}
}
LOG.debug("Done.");
}

}


AWS: How do you subscribe an SQS queue to an SNS topic?

Image via CrunchBase

If there are any AWS (Amazon Web Services) gurus out there, I'm stumped on trying to use a Simple Queue Service queue to receive notifications sent out on a Simple Notification Service topic. Any pointers?

In the code below, I'm using the AWS SDK for Java (available here). The program does the following:

1. Create an SNS topic.
2. Set a policy on the topic to allow subscriptions.
3. Create an SQS queue.
4. Set a policy on the queue to allow message posting.
5. Subscribe the queue to the topic.
6. Send an SNS notification message.
7. Listen for the message on the queue.

Steps 1-6 seem to work just fine, but on step 7, it listens in vain for messages. All ten iterations go by without picking up any messages on the queue. As an added note, if I subscribe an email address to the topic, messages are delivered. Only the queue appears to be bypassed when SNS sends its notifications.


public class SnsWithSqsDemo {
private static final String TOPIC_NAME = "TestTopic";
private static final String QUEUE_NAME = "TestQueue";

/**
* This program demonstrates how to hook up an AWS SQS queue to an AWS SNS
* topic to receive messages.
*
* 1. Create an SNS topic.
* 2. Set a policy on the topic to allow subscriptions.
* 3. Create an SQS queue.
* 4. Set a policy on the queue to allow message posting.
* 5. Subscribe the queue to the topic.
* 6. Send an SNS notification message.
* 7. Listen for the message on the queue.
*
* @param args not used
*/
public static void main(String[] args) {
AWSCredentials awsCredentials = null;
AmazonSNS sns = null;
AmazonSQS sqs = null;
String topicArn = null;
String queueUrl = null;
String queueArn = null;
String subscriptionArn = null;
try {
awsCredentials =
new PropertiesCredentials(
SnsWithSqsDemo.class.getResourceAsStream("AwsCredentials.properties"));
sns = new AmazonSNSClient(awsCredentials);
// 1. Create a topic
CreateTopicResult ctResult =
sns.createTopic(new CreateTopicRequest(TOPIC_NAME));
topicArn = ctResult.getTopicArn();
System.out.println(String.format("Created topic %s with ARN %s",
TOPIC_NAME, topicArn));
// 2. Set policy on topic to allow open subscriptions
Policy snsPolicy =
new Policy().withStatements(
new Statement(Effect.Allow)
.withPrincipals(Principal.AllUsers)
.withActions(SNSActions.Subscribe));
// ??? Is there no Java SDK constant for "Policy"?
sns.setTopicAttributes(new SetTopicAttributesRequest(
topicArn, "Policy", snsPolicy.toJson()));
// 3. Create a queue
sqs = new AmazonSQSClient(awsCredentials);
CreateQueueResult cqResult =
sqs.createQueue(new CreateQueueRequest(QUEUE_NAME));
queueUrl = cqResult.getQueueUrl();
System.out.println(String.format("Created queue %s with URL %s",
QUEUE_NAME, queueUrl));
// 4. Set the queue policy to allow SNS to publish messages
Policy sqsPolicy =
new Policy().withStatements(
new Statement(Effect.Allow)
.withPrincipals(Principal.AllUsers)
.withActions(SQSActions.SendMessage)
.withConditions(ConditionFactory.newSourceArnCondition(topicArn)));
Map queueAttributes = new HashMap();
queueAttributes.put("Policy", sqsPolicy.toJson());
sqs.setQueueAttributes(new SetQueueAttributesRequest(queueUrl, queueAttributes));
// ??? Why isn't the queue ARN in the queue attributes?
AmazonIdentityManagement idMgr = new AmazonIdentityManagementClient(awsCredentials);
String accountId = idMgr.getUser().getUser().getUserId();
queueArn = String.format("arn:aws:sqs:us-east-1:%s:%s", accountId, QUEUE_NAME);
// 5. Subscribe the queue to the topic
SubscribeResult sResult =
sns.subscribe(new SubscribeRequest(topicArn, "sqs", queueArn));
subscriptionArn = sResult.getSubscriptionArn();
System.out.println("Subscription ARN: " + subscriptionArn);
// 5.5. Wait a bit for AWS to get all synched-up
Thread.sleep(60000L);
// 6. Send a notification
PublishResult pResult =
sns.publish(new PublishRequest(topicArn,
"Mr Watson -- Come here -- I want to see you."));
System.out.println("Sent message ID = " + pResult.getMessageId());
// 7. Wait for message receipt in queue
for (int i = 0; i < 10; i++) { Thread.sleep(2000L); ReceiveMessageResult rmResult = sqs.receiveMessage(new ReceiveMessageRequest(queueUrl)); if (rmResult.getMessages().size() > 0) {
// A message has been received
for (Message message : rmResult.getMessages()) {
System.out.println(message.getBody());
sqs.deleteMessage(new DeleteMessageRequest(queueUrl,
message.getReceiptHandle()));
}
break;
} else {
System.out.println("No messages available, attempt " + (i+1));
}
}
} catch (Exception e) {
e.printStackTrace(System.err);
} finally {
// Unsubscribe the queue from the topic
if (sns != null && subscriptionArn != null) {
sns.unsubscribe(new UnsubscribeRequest(subscriptionArn));
System.out.println("Unsubscribed queue from topic.");
}
// Destroy queue
if (sqs != null && queueUrl != null) {
sqs.deleteQueue(new DeleteQueueRequest(queueUrl));
System.out.println("Deleted the queue.");
sqs.shutdown();
}
// Destroy topic
if (sns != null && topicArn != null) {
sns.deleteTopic(new DeleteTopicRequest(topicArn));
System.out.println("Deleted the topic.");
sns.shutdown();
}
}
}

}


2011: My Android Year

Image via Wikipedia

Gadget journalism on the Web seems to be held to the same standard as entertainment journalism, meaning rumors, guesswork and hype make up most of my Google News alerts and Twitter feeds. But (like entertainment journalism, to those who follow it), the hype is strangely addictive. And the hype isn't just about a new device hitting the shelves, but upcoming announcements about devices that won't see a shelf for months, or even announcements about upcoming announcements. It starts with leaked device code-names or Photoshopped concept images, then there are unconfirmed spec sheets and rumored release dates, followed by official teasers and then trade show releases (with the hands-on videos and device comparison sheets). Finally, YouTube starts getting the gadget version of amateur porn: the unboxing videos. It's an information strip-tease, showing just enough to catch the attention and drive the imagination wild with what isn't revealed.

I'm not an early adopter of technology, and when something starts to get exciting, I research it for months before I make the jump. I've been carrying around a Windows Mobile phone (the HTC Touch Pro 2), top-class technology that integrated well with work when I bought it, but it seems that Microsoft as a mobile platform is moribund, while Android is the new market share powerhouse. In early 2010, I started following rumors of a beast of an Android phone called the HTC "Scorpion" and for the next year, I submerged myself in Android hype. By January's Consumer Electronics Show, I was sold on two or three Android devices scheduled to release in the first half of the year.

I'd followed a rumored Motorola Android tablet code-named "Stingray," which was introduced at CES as the "Xoom." The Xoom is the hardware reference for Google's new release of Android, version 3.0 dubbed "Honeycomb," and it boasts a dual-core processor, rear- and front-facing cameras for video chats and HDMI output among its features. When the iPad came out, I couldn't understand where it fit in the gadget ecosystem: was it a bulky smartphone (sans calls), an underpowered laptop or a high-eyestrain eBook reader? Then, as I moved many of my computer activities into the cloud, live-blogged from geek conventions and trips, and grew weary of taking my laptop out of my carry-on at airport security, I began to realize that the tablet was the ideal travel laptop. The more I researched the applications and accessories (like a Bluetooth keyboard) that would be available, the more the tablet's niche solidified:

• Live-blogging
• Travel laptop that stays in the carry-on, through security
• Garmin navigator replacement
• Stream media (Netflix, Hulu, etc.) to any TV
• SSH into my work computers and Amazon EC2 instances
• Portable movies
• Document editing (with Google Docs)

As of this writing, the Xoom is supposed to release on February 24, 2011. But that information comes from the gadget rags: Motorola and Verizon are silent on the topic.

As for the Android phone, up until CES 2011, I was looking forward to the HTC Thunderbolt, billed as "Not your dream phone. The one after that." It's sleek like the Nexus One, thin but with a huge 4.3" screen. It's the first "4G" phone on Verizon's LTE network. But Motorola came to CES with a surprise: the heretofore un-hyped (and unheard of) DROID Bionic, with a dual-core processor (the Thunderbolt's is single-core), more on-board storage and an HDMI-out port. The Bionic's case is not as pretty as the Thunderbolt's, but it seems to be more of a powerhouse. Even if Android 2.2 doesn't utilize dual core processors (and that's a big IF -- I haven't seen it categorically demonstrated that this is the case), Android 2.4 will, and software always evolves to push the hardware. The Bionic will have staying power. My wife surprised me by turning down an iPhone to replace her dying BlackBerry Storm, so I'm going to get her the Thunderbolt and stick with my Windows Mobile phone until the Bionic comes out. I'll be able to do some real comparisons then.

Now, I wait. Neither Verizon, nor HTC nor Motorola have made an official statement about release dates, but the current rumor-mill has the Motorola Xoom releasing on February 24 2011, the HTC Thunderbolt releasing on February 28 2011, and the Motorola DROID Bionic releasing in June 2011. After that, I'll have to wean myself off of Android hype news, and enjoy what I have instead of lusting after the next cool thing.

Amrut Fusion: the acclaimed single malt whisky from... India??

Whisky writer Jim Murray named Amrut Fusion Single Malt the world's third finest whisky, giving it 97 out of 100 points. What's interesting is that Amrut Fusion isn't a product of Scotland, or Ireland or even Kentucky. This whisky comes from Bangalore India, a city known for outsourced information technology rather than fine spirits. As a friend put it, "Beating the English at Cricket or Soccer, well that's just fun... but beating the Scots and Irish at booze... that I have to taste!"

As did I. And this evening, my wife came home from work with a surprise for me. The local Binny's Beverage Depot carries the entire line of Amrut products. "Amrut" in Hindu mythology is the nectar of the gods. I've been looking forward to pouring a glass and sharing the experience with you. Let's begin!

This whisky is a lovely light amber color, and its aroma hits you right as you pour it: the first whiff reminds me of cedar and cherry. It's a very fresh profile, but if you stick your nose in the glass like a wine taster, you can smell an undercurrent of smoke and maybe a hint of peat.

My first sip bites at the tip of my tongue like a super-hot chutney. This is not a whisky I would typically drink neat. It's rough on the tongue, but full of flavors of fruit and vanilla-oak, with cloves and orange rind simmering at the back of the throat after it goes down. I'm not well-versed in whisky, but I've tasted a fair bit of wine, and Amrut Fusion has the same level of complexity I'd find in some of my favorite wines. It changes in your mouth, from the sharp, sweet attack at the tip of the tongue, to a spicy, smoky robustness at the end.

The Indian men of my parents' generation love their Canadian Club, and get excited when they get their hands on a bottle of Chivas Regal. Now, they can drink something light years ahead of their old standards. And best of all, it's from their homeland. Amrut, you've made a drink worthy of the name.

Related articles

• Whisky Bible Awards Ballantine's and Buffalo Trace Top Three 2011 Honors (luxist.com)
• Properly Sauced: Amrut "Fusion" Single Malt Indian Whiskey (chicagoist.com)