For other “more” technical posts, we suggest our Dev blog.
Recently here at FreshBooks we’ve been busy re-factoring and updating how we serve content delivered through XHR or ‘ajax’ requests. There were several factors and reasons that drove us to rework a subsystem that has worked well for the past 5 years.
- Difficult to test – The current implementation has a number of factors that make it challenging to test.
- Difficult to extend – Other details with the current implementation cause it to be more challenging to extend than we’d really like.
- Coupled to XML – Xml is great, but JSON is a smaller, and for our uses easier to work with format.
Once we knew the pain points of our current system, we started looking at what our ‘ideal’ implementation would look like, and ways that we could incrementally migrate code into a codebase that would be easier to test and allow us to use non-XML data formats.
Planning the solution
After collecting the problems we wanted to solve, we started to plan out the new approach. We started out by whiteboarding potential solutions and the object structure we wanted. Using our list of what was wrong with the current implementation as a guideline of things we wanted to avoid, we discussed the structure we wanted and drew out possible implementations on a whiteboard. After getting something that looked good in sharpie, our next step was to hack out a prototype using test cases to make sure it would do what we want. We presented this prototype to the rest of the team to get feedback and improve the design.
In the end we went with a structure that provided separation between serialization, domain logic, and request handling. This not only made the new code shorter, but also allowed us to write more specific and more accurate tests. This separation also allowed us to reduce a lot of duplication that was present in the existing implementation.
Once we had planned out how we wanted to tackle the problem, and our prototype looked feasible, we polished up the prototype and continued to implementation. New code was created with unit and integration tests. At FreshBooks, we have a 2 ‘ajax’ endpoints. One handles non-account-specific features, and the other handles per-account features. In the old system each file contained all the functions it needed, which meant duplication was just a fact of life. For the new object-based implementation we didn’t want to maintain this physical separation in the code base. A physical separation like the previous implementation would introduce additional unwanted complexity. However, we still needed an explicit way to declare which methods could be exposed where. As part of our planning process we decided to take a page from PHPUnit and used method annotations to declare under what context a method could be used. The two zones were named ‘inside’ and ‘outside’, for their roles inside and outside a customer’s account. Every method would need to have an
@expose doc tag that indicated under what context it could be called. Methods without this annotation would not be accessible from a URL — this would help prevent developer error and leave things in a ‘deny by default’ mode.
We were able to solve the testing and serialization separation issue by following MVC ideas. By separating the logic and serialization steps, we were able to more easily test the logic and provide extensible serialization. Instead of having to write tests against serialized output we could write tests against data structures, which is much easier.
Incremental rollout into production
After the initial set of tests were written for the common libraries, we decided to use a low-risk area of the application to spike our solution and make sure it was practical and functional. The current implementation at its heart is a very long case switch. Because of this we were able to slowly ‘strangle’ out the old code and replace it with new code. We put the new request handling at the bottom of the switch so it would be able to catch any requests the existing code could not. If a request falls through the case switch, it means that the code has either been cleaned up or the request is bogus. The new system attempts to handle these left-over responses with its methods. This is a huge win, as it has allowed us to incrementally weed out the old implementation. Each section can be extracted out of the old implementation and moved into the new. This can be done safely and incrementally, so that we don’t overburden development or QA with our updating efforts. This ‘strangulation re-factor’ approach has allowed us to iteratively and safely roll out the new codebase with no disruptions.
I have to say I’ve been very happy with our approach and process. During the entire process we have had no customer-reported issues with the new implementation. Unless you spend the time to dig through the network traffic, you won’t even know that we’ve been overhauling what a pretty important piece of the FreshBooks experience.