Site of the Month: Better Hotel Room

December’s ‘Site of the Month’ – Better Hotel Room (BHR) – is a multi-region set of sites developed by SilverStripe Professional Partner Internetrix. The Australian digital company are experts at website projects, online performance and digital consulting.

This ‘Site of the Month’ is a story about an incredible journey that the Internetrix team went through to build a powerful site that caters to a big audience across different countries. It will give you many valuable lessons in multi-region SilverStripe deployment.

Background

BHR operates an online hotel booking system covering thousands of quality hotels available in Australia, New Zealand, Hong Kong and around the world on both Australian and Chinese websites.

Putting themselves into play as the guinea pig, the very first website Internetrix deployed into mainland China was their own. This attracted interest from BHR who came to them with an initial project scope to sell about 2,000 AU and NZ hotels to an Australian and Chinese market.

After Internetrix took a working website to BHR who showed it to their investors, they decided to go further, go bigger, go global.

Challenges

Now that the site was going global, it would need to offer over 125,000 properties from all over the world (which is a very different proposition from the original 2,000) and deploy more websites in other regions.

These significant changes increased the challenges considerably:

1. The Currency Challenge

Starting with Australia, a single country with a single currency, Internetrix looked to employ code based on SilverShop, which uses a single currency.

This relatively calm situation exploded when they were faced with the new global scenario:

• 31 available currencies
• Hotels come from all over the world, each returning prices in their native currency
• User can choose a preferred currency

Their answer was to overhaul the currency approach to:

• Convert the price into the base currency
• Associate the native currency to the hotel
• Convert to user-selected currency when displaying

One snag was that SilverShop only supports one currency, so they needed to convert when saving the order. Another snag was that the Hotel-reported currency didn’t always match the hotel country’s currency or their actual price currency.

Internetrix put on their thinking caps and came up with their own Currency Module. It uses YAML files for storing currency information and they then save each rate conversion to the database.

• Gets 31 exchange rates from Fixer.io API
• Fixer.io rates are updated from European Central Bank once daily, but they only update the site once a week
• They mapped currencies to country & locale using data from LocalePlanet

2. The Translations Challenge

The website targets Australian and Chinese markets, so the natural next step was to look at translation. Chinese script is used differently in different regions, so Internetrix needed to decide between Simplified Chinese (Mainland China and Hong Kong) as opposed to Traditional Chinese elsewhere.

Their first three moves to support translations was to:

• Use the Fluent module
• Write their own Fluent extension
• Use Language YAMLs for design elements

Internetrix choose not to use Translatable as it creates a unique tree for each language, in turn creating more work to maintain; and as it uses URL parameters, which was undesirable.

Instead Internetrix choose to use the translation module Fluent. However, store-by-columns raised issues, which they solved by making a SilverStripe Fluent Extra Table. Each language gets its own table whereby there is 1 extra left join. You can find their solution on GitHub!

Working with YAML translations in SilverStripe meant they could combine YAML syntax, PHP syntax, and template syntax which avoids rewriting code for different languages. The advantage is being able to easily drop in new languages.

3. The Regional Challenge of Deploying in China

Deployment in Australia is straightforward and familiar to Internetrix, however deployment in China is more complex. So, the first question is why would you want to host in China? For the BHR project, the goals included:

• To target a Chinese market –  the main reason for the BHR project
• To improve website availability and trust – need to have website as fast, available and trusted as possible by a Chinese audience
• To reduce server latency – having the location of the server closer to your targeted audience will reduce server latency for them

However, the prime reason was to bypass The Great Firewall of China, part of the Golden Shield program initiated by the Chinese government in 2003, to affect the censorship, monitoring and filtering of internet content from foreign websites.

The key prerequisite to do this is to apply for and receive an  Internet Content Provider (ICP) licence. The application process needs determination and patience to follow the steps to successful approval:

• Step 1: Site owner submits details to ISP (AWS China)
• Step 2: Site ownership verification
• Step 3: ISP submits application to MIIT*
• Step 4: Data enters provincial MIIT System
• Step 5: Provincial MIIT branch checks application
• Step 6: MIIT notify ISP with feedback or approved ICP number
• Step 7: ISP notify site owner

* MIIT – Ministry of Industry and Information Technology

4. The AWS Challenge

Within China, AWS is different to the rest of the world with a limit product offering. AWS China is only available to those who:

• Apply for an account
• Are a China legal entity

The next hurdle was that Internetrix found data speed varies by region...and in and out of Beijing it was extra slow. For example, rather than taking the route Sydney > Beijing, they actually found out that Sydney > Singapore > Beijing was the fastest route.

Here are some figures to download a 67mb file:

5. The Technology Challenge

The technology landscape in China is very different, not least because everyday names outside China do not work there, including the following:

• Google Maps
• Twitter
• Facebook

This is where Internetrix needed to dig hard with our External Authentication module. There are a number of external authentication modules available but Internetrix chose opauth. This is an awesome module that makes it very easy to authenticate with a number of different providers. You simply need to define an OpauthStrategy, plug in the necessary credentials, and away you go. What’s even better is that most of the popular authentication providers already have strategies written for them.

Unfortunately, Facebook authentication does not work in China. Internetrix decided to integrate with China’s very popular micro-blogging social media site called Weibo. Luckily, getting this to work was quite simple (Internetrix wrote the Weibo opauth strategy, which is available on GitHub).

6. The Regional Latency Challenge

Having found that the geographical location of servers is important, Internetrix then bumped up against another lesson, that the location of third party systems is also extremely important.

Key to working with BHR was Internetrix’s knowledge of a GDS or Global Distribution System. This is the link between travel service providers (hotels) and travel agencies (like BHR). There are a number of popular GDSs, for example, Amadeus, Pegasus Solutions, and TravelPort. However, BHR had already chosen to go with Sabre that links to thousands of different hotel customer reservation systems (CRS).

The snag Internetrix found was that Sabre is only located in the USA, which meant it was slow when searching Australian hotels. The following are some average Sabre call response times:

Step 1 Hotel Availability 8-10s
Step 2 Hotel Property Description 3-4s
Step 3 Hotel Rate Description 3-4s
Step 4 Book Hotel Reservation 3-4s

Always ready for a challenge, Internetrix made their first attempt at a fix using organic cache. This meant that they simply saved the search results to a cache until a certain expiry time. Unfortunately, this meant that the first user still had to wait for the response from Sabre, but every result after that, for the same city, check-in, check-out date etc would load fast.

This also had the advantage of the site appearing to load faster, as more and more people used it and generated cache results. But this still wasn’t good enough as the first user still had to wait for too long.

They went back to the drawing board. They decided to make assumptions and pre-generate cache results for popular cities and common date ranges. But running a task to generate all these cache results would bring the website to a halt.

So how could they do this? The answer was Queued Jobs, which is an awesome module that lets you define long processing tasks as jobs, such as the BHR ones. These ‘jobs’ can then be added to a queue, which can then be processed asynchronously as a background task. You simply setup a cronjob to process the queue.

Internetrix identified about 71 popular cities. In order to generate cache results for 71 cities for common date ranges, they then calculated the processing time (which includes all the wait time from Sabre) to be 174 hours.

It was clear that Internetrix would need to distribute this processing between multiple threads and multiple servers. API slowness meant they needed to run many in parallel. With the queued jobs module, they could use different engines to process the queue.

They decided to use the Doorman Runner (multi thread execution) included in the module by default, which allowed them to enable multi process execution.

7. The Pricing Challenge

Even after all of this heroic action, the next logjam was that the pricing was still wrong! Pricing that came back from the hotel availability call was the minimum price. But the minimum price might not be available. So when users clicked into the hotel to view the different available room rates, there might be a difference.

Internetrix decided to fire off an AJAX call for each hotel in the search results.

But then they found another issue: a connections limitation. So it turns out that there is an upper limit for the number of concurrent connections a browser can make to a hostname.

In a list of 20 hotel search results, they would have to wait.

So what did Internetrix do? They built a server cluster behind an Elastic Load Balancer (ELB). Their BHR solution was to use fake subdomains to update prices in the following steps:

Step 1: Deployed a server cluster behind an Elastic Load Balancer (ELB)
Step 2: Each price lookup uses a different subdomain, which all point to the ELB
Step 3: Price lookup result is cached

Finally, they implemented the Distributed Parallel Requests module, which gave them these advantages:

• Replaces relative CSS, JS and image links with distributed absolute links to trick the browser
• Can point to external CDNs

Conclusions

So what are Internetrix’s key learnings from building and deploying the awesome BHR Hotels website in China?

Currencies

• Convert to the “base” currency for processing – Keep versions of currency rates (or include rate in order object?)
• Always display money as a conversion – Write code for conversion even if converting to same currency
• Add currencies without needing to rewrite code – Rely on YAML for adding currencies

Translations

• Utilise YAML strings wherever possible, right from the start – Don’t be tempted to throw in hardcoded strings, they’ll need replacing
• Pick the translation module that suits your needs.
• Database column-length limitation when using Fluent

Regional Limitations

• Transfer speeds vary, look for alternate paths – Alternate paths and processes – Don’t always need to write a code solution
• Don’t assume specific technologies are available or identically offered – Be aware of local technology available
• Watch out for non-technical obstacles (eg. Chinese ICP license process is very slow)

Regional Latency

• A global service provider does not mean all regions are equally served – Don’t underestimate latency… even from global companies
• Local precaching overcomes latency – Precaching with silverstripe australia queued jobs module
• Distributed delivery ensures data is fresh & fast

Innovate

• Currency conversion module
• Fluent extensions module
• Distributed parallel requests module
• Weibo OpauthStrategy module

Getting the BHR websites up and running was part obstacle race, part ingenuity test and well worthwhile. Throughout this journey, Internetrix found that SilverStripe is a fantastic framework that helped them create solutions that are flexible, extensible and sensible.