re:Invent 2017 – Building a Mission Critical, Serverless Photo Pipeline for 100 Million Photos
Monday, November 27th 2:30 PM, Aria, Level 3, Juniper 4
Mike worked on HomeAway‘s image processing pipeline using Lambda functions, SNS, and DynamoDB to handle an ever growing collection of over 100 million property photos uploaded by the property owners. He’ll share the from scratch evolution of the service, walking the audience piece by piece through the overall architecture and sharing some of the key insights. You’ll learn:
- Where Lambdas work and where they don’t
- Best practices for limiting Lambda processing time,
- Some surprising results about Lambda memory optimization,
- How important using the dead letter queue is to reliable and scalable pipelines.
Abstract: In this session, principal architect Mike Broadway describes how HomeAway built a high-throughput, scalable pipeline for manipulating, storing, and serving hundreds of image files every second with Lambda, Amazon S3, DynamoDB, and Amazon SNS. He also shares best practices and lessons learned as they scaled their mission-critical On Demand Image Service (ODIS) system into production. Lambda functions form the backbone of ODIS, which handles over 100 million photographs that are uploaded to HomeAway’s vacation rental platform. HomeAway is a vacation rental marketplace with more than 2 million rentals in 190 countries and is part of Expedia.
See the complete list of Expedia team members speaking at re:Invent 2017.