An ideal summer internship at Goibibo
Editorial note: Elita interned with Goibibo few months ago. Recently she expressed her experiences on her blog. We are reproducing it here as-is. Some of her work is part of a larger release cycle, and isn’t available on Goibibo at the time of posting this.
Hello Everyone, It’s been almost 3 weeks since my internship at GoIbibo has ended and I thought what could be better than writing a blog to preserve good memories?
So here it is, a full description of how I spent my summers hacking into some cool projects at GoIbibo!
Day 1:
I was excited to work in a company which had developed rapidly in a few years span! This was my first internship in a well-established start-up and I was hoping that I could make the best out of it. I was introduced to my mentor Bala Phani Chand (Principal Software Engineer at GoIbibo) by the HR. Phani told me that I would be a part of the control team which is responsible for Backend Development and consists of some of the brilliant minds of GoIbibo!I couldn’t wait to get started on my project! :D
My First Project:
Before getting into the technical details of my project, let me ask you, how many times were you disappointed due to sudden change in price of flights when navigating from search result page to flight booking page? I bet you must have thought it is a technique of attracting customers but it is not! GoIbibo used to cache the results for a fixed short duration of time to avoid unnecessary hits to the API. So GoIbibo decided that it was time to fix this problem in such a way that the Customers are happy and they are happy too! :) My job was to devise an intelligent algorithm which will reduce the no of such invalidations occurring specially during sales and some special occasions when the demand is high. Sounds interesting, doesn’t it?
The Birth of CacheBot:
I began researching on various factors which lead to increase in the no of invalidations and relation between these factors. I learnt machine learning algorithms as well as implemented them to predict the time for which the data is to be cached. The accuracy obtained was around 80% which wasn’t quite satisfactory. I then implemented multi-layer neural networks which failed to improve the accuracy and also handle cases when there is any sale. I used to constantly discuss the results of my research with Phani and Mr Neeraj Koul (Director of GoIbibo) and they would always help me figure out a better approach to solve the problem. After a whole month of research with various algorithms, I was successful in developing a Cachebot which determines the time for which the new data is to be cached based on past data results (obtained using machine learning algorithms) and current scenario analysis. The accuracy obtained was around 95% and the best part was it could handle sales as well as any sudden change in demand! :D
GoIbibo 24 hours Hackathon:
GoIbibo conducted an awesome hackathon in which only its employees and interns were allowed to participate! We had access to unlimited food and drinks! :D All the teams coded the entire night and tried to compete with the other teams. I was amazed at the awesome products developed by each team in one night! Btw my team included Neha Jagadish , Pawan Sharma (Two of the finest backend developers from the Flights team) and Rupesh Singh (a talented JS developer). We developed a price tracker which instantly notifies the Customer when the prices of the flight he/she is tracking has changed! We managed to come up with a constant time solution for the same using some simple hacks on database signals! :)
Here is a group photo of all the GoIbibo developers who participated in the hackathon! :) These guys were still bubbling with enthusiasm even after spending the whole night coding! :D Kudos to the awesome GoIbibo Devs! (Y)
Project 2:
My second project was another interesting and challenging problem! :) The aim of this project was to write a distributed cache library in Go which would store data in the form of key-value pair in-memory of each node and thereby reduce the latency which is generally encountered in Memcached or Redis. But you must be wondering why write it in Go instead of Python? The major reason we used Go is because of its awesome concurrency feature and its channels which are greatly useful for reducing delay in networks. I was mentored by some of the most intelligent minds of GoIbibo — Mr Jyotiswarup Raiturkar who is the Head Architect of GoIbibo, Neeraj Koul and Harshad Patil (Senior Software Engineer).
GoCache:
This project required several discussions on the design and constant modification of the design! I encountered many challenging problems while implementing this library which I used to discuss with Mr Jyotiswarup, Mr Neeraj and Harshad, and they would help me figure out efficient solutions to those problems. Although I was an intern, I was free to put forward my ideas or challenge their ideas without having to think twice! :)
By the end of my internship, I had finally implemented a distributed peer to peer network wherein each node has its own BusSocket which can listen to peer nodes, dial to peer nodes, receive messages from peer nodes and evaluate them one at a time and also send messages to peer nodes concurrently. Akansha Gupta wrote an api for Ledisdb-Goleveldb cache which I used for maintaining a storage for each node. GoLeveldb is super fast, has expiry feature and allows concurrent reads and writes in a single process, and that is why we chose it for our library. I also implemented automatic redialling until connection is established, when connection is lost and EOF is received on the connection. Each node could send acknowledgement to its peers after evaluating the message received. If a node fails to receive acknowledgement of the message it sent within the timeout period, it would break the connection and redial to the peer. Harshad implemented stale data error acknowledgement which notifies the sender node that the data it sent is stale and updates the sender’s node database with the most recent data. We also implemented an optimal solution for handling cases wherein a node’s peer goes down for a while and by the time it comes up, its database is outdated.
On testing the code, we found that the network could send and simultaneously receive 1 million messages to and from its peers in 3–4 minutes. Yay! :)
My mentor Harshad has done a great job repainting the code, optimizing it and encapsulating it. The code has now been made open source and pushed to the GoIbibo github. Anyone and everyone is welcome to contribute to it. :)
Link to Harshad’s GoCache Library: https://github.com/goibibo/stash
Link to my library which is still under development : https://github.com/elitalobo/DistributedCache
Overview of my internship experience:
I enjoyed every bit of it, so much that I wished I never had to return to college! :D I made really great friends and we had a tons of fun together! :) The timings were flexible. I had all the freedom to voice my ideas and and they were always considered! And the best part of the internship is that I got to learn a lot from my projects and my awesome mentors which I doubt I could have learnt elsewhere! :) Summer was well spent! :)
Originally published at goibibo.github.io on August 14, 2015.
