Don’t try this at home (just get Embedly)
Launching Banterly last week was a thrill. It was unbelievably fun meeting so many new people from all over the world through my own personal creation. Everyone was so friendly and supportive and I honestly loved every minute of it (even the parts where I was frantically fixing bugs). It was an experience I won’t soon forget and hope to repeat.
Not long after I launched, the guys at Embedly started a discussion suggesting that I check out their oEmbed service. The moment I saw it, I knew it was exactly what I had been looking for. The only annoying part was that I hadn’t discovered it several weeks ago when I first started working on embedded media in Banterly. So, for that reason, I decided to do a little demonstration and offer a little praise in hopes that others looking for a solution to their embedding woes might not waste time trying to roll their own (like I did).
Don’t DIY
Let’s start with what NOT to do. Don’t try to do media embedding yourself. For starters, there are simply too many different sites out there, each with slightly different methods of exposing their data (Embedly supports more than 160 providers for free or 185 for a reasonable subscription fee). Doing it myself, it took two days to get support for images, YouTube, and the most simple OpenGraph meta tags (which was extremely flimsy since no complete HTML parsers exist for Node yet).
Additionally, doing it yourself means downloading unknown content from unknown sources at the whim of your users—not exactly the kind of thing you want to expose yourself to when all you want is to embed content on your site. In fact, I almost cut the feature before launch because I was afraid malicious users might discover it and use it to abuse the system, but then I thought—‘why not?’ and left it in anyway.
embedly-node
You can use Embedly to embed media from both server side and client side (via jQuery). I chose to use the server side module (embedly-node) in order to reduce the number of requests I have to make to the Embedly API and to keep page load times fast for users. Using the embedly-node module on GitHub (use npm install embedly), I ended up with something similar to the following:
var Embedly = require("embedly");
var embedly = new Embedly.api({
//key: "XXXXXXXXXXXXXXXXXXXXXXXXXXXXX", // Embedly Pro Key
user_agent: 'Mozilla/5.0 (compatible; myapp/1.0; me@example.com)'
});
// Do initial comment text processing
// 1. Parse URLs from comment text
// 2. Render HTML comment by replacing parsed URLs with anchor tags
// 3. Save rendered comment HTML to the database
// 4. Send rendered comment HTML to connected clients (via WebSocket)
...
// 5. Try to get additional content from Embedly
if (urls) {
// Using Embedly oEmbed API
embedly.oembed({
params: {
urls: urls // Can send single URL also.
},
complete: function(e, objs) {
for (var i in objs) {
// Returned if the URL content is not supported on free tier
if (objs[i].type == "error") continue;
// Some embed types don't contain a URL
objs[i].url = objs[i].url || urls[i];
// Render and replace the previously rendered link
html = html.replace(getLink(urls[i]), render(objs[i]));
}
// Only save and send to clients if the HTML has changed
if (comment.html != html) {
// Update the rendered HTML in the database
comment.html = html;
comment.save(function() {
// Send updated comment HTML to connected clients (via WebSocket)
...
});
}
});
}
Sending the initial HTML to connected clients immediately (rather than waiting for Embedly) helps keep the UI fast and responsive while users are chatting. Once Embedly returns, new HTML is sent out to clients to replace the initial comment markup. In my tests, I found Embedly to be quite fast at returning embed data, but its speed will always be limited by how fast the URL host can return its own content, so it’s much more reasonable to keep the updates separate.
A simplified version of my render function looks like this (would be admittedly much cooler with a template):
function render(obj) {
if (obj.type == "photo") {
// Just an image (only works with pro account)
// image dimensions are available as obj.height and obj.width
var html = "<img src='"+obj.url+"' width='500'/>";
return html;
} else if (obj.type == "video") {
var html = "";
// Add thumbnail
if (obj.thumbnail_url)
html += "<a href='"+obj.url+"' target='_blank'><img src='"+obj.thumbnail_url+"' width='100'/></a>";
// Add title link
if (obj.title)
html += "<a href='"+obj.url+"' target='_blank'><h6>"+obj.title+"</h6></a>";
// Add an explicit link
html += "<a href='"+obj.url+"' target='_blank'>"+obj.url+"</a>";
// Add description
if (obj.description)
html += "<p>"+obj.description+"</p>";
// Wrap it up
return "<div class='oembed'>"+html+"</div>";
}
// Just a normal link
return "<a href='"+obj.url+"' target='_blank'>"+obj.url+"</a>";
}
In addition to oEmbed, there are also ‘preview’ and ‘objectify’ which I have yet to explore (maybe I’ll follow up if I do) but are only available for Pro Embedly users.
Thanks guys!
I also think it’s worth mentioning how helpful the folks at Embedly were during the process (especially Bob Corsaro aka @doki_pen). Throughout the experience, they were extremely available (chatting on Banterly and exchanging emails late into the night). They even fixed a bug I found within minutes of reporting it!
