eng.age

From Angular to Ember

Ember.js Tomster

In October 2015, the UI team at Verizon Digital Media Services started to reenvision the next generation of the Media Control Center, our customer facing CDN portal. The MCC was started in 2008 and is a traditional server-rendered application built on the .NET technology stack. Over the years, the front-end started playing a more prominent role using JavaScript technologies including jQuery, Angular 1, Gulp and CoffeeScript (popular at the time). Even though modern front-end tech was used, there were several pain points that frustrated front-end developers, leading the team to start thinking about how we wanted to build out MCC. Should we continue with Angular 1 and hope to switch to Angular 2 if there is an upgrade path, or should we switch to something else?

Continue Reading

Serializing to Parquet from Kafka with Exactly Once Guarantee

In the process of building our new analytics pipeline, we had to implement a typical lambda architecture. A lambda architecture is where you need to implement one massive process flow for real-time data and another for batch for the sake of performance. Mainly because the optimization points for real-time and batch are almost entirely unique to each other. Throughout our process of designing and improving the batch portion of our new lambda architecture, we faced many challenges and learned helpful lessons. We hope a summary of these takeaways will offer helpful insight to those implementing their own batch pipelines.

Continue Reading

Riding the Riptide

The Problem

Normally when we talk about traffic at a CDN, we think about pushing data out to customers. While this seems like it should just involve straightforward transmissions, it turns out things can be a bit more complex.

Continue Reading

Malloc Lives In Userspace (and why you care)

Python, Ruby and Java handle memory management for us. Containerization and “serverless” operation let us do more and more on a single server, and work at ever higher levels of abstraction. But they don’t actually make the server underneath go away and the nitty gritty details of kernels, memory allocation, and storage can still ruin your day. Learn a few lessons about memory based on experiences running a CDN at a scale large enough to serve Tbps of traffic.

Continue Reading