A Lesson from Uber: Secure Your Non-Production Software Environments

Earlier today, the FTC announced a revised settlement with Uber regarding the company’s privacy and data security promises. The case involved multiple breaches of Uber’s cloud storage infrastructure where the company stored full and partial backups of databases containing information about Uber users and drivers. The complaint alleges that Uber failed to reasonably secure this cloud storage. According to the complaint, Uber software engineers would develop and test software that could connect to this cloud data using inadequate access controls. As a result, intruders exploited Uber’s software development environment to take advantage of the company’s failure to reasonably secure its cloud storage. This case demonstrates the importance of securing all software environments, not just production environments.

Companies often focus their privacy and data security efforts on the up-and-running production environments actively used by their consumers. Such systems deserve attention because a breach of such systems can damage the company and its consumers. In addition, production systems often contain the most valuable data, making them attractive targets for intruders. But an insecure software development environment can also create real problems. Insecure non-production environments leave a company open to corporate espionage, sabotage by competitors, and, yes, theft of private consumer data.

Many data security best practices (such as those discussed in the FTC’s numerous resources) apply in both production and non-production environments. But securing a software development environment may require a different emphasis because such environments present at least four challenges:

• Different Convenience / Security Tradeoffs. Development environments simplify the complex job of writing software. Like a bank under construction, software in development doesn’t have all the security features that will be in the final product. Additionally, software developers often need broader access to data and functionality during development than for production software. Attempting to impose internal restrictions during development could impede progress without adding much security. (Imagine requiring all doors on a construction site to be locked, when the walls aren’t yet constructed.) Thus, development environments often relax internal restrictions. Instead, development environments depend heavily on access controls to ensure that only authorized developers can enter the environment – like fences around a construction site. When done properly, such environments can provide security while simplifying development. However, sometimes the relaxed internal environment can lead to complacency in the access controls.
   
• Use of Test Data. Software developers must test both the functionality and security of their software. To properly test a system, one needs data, but generating appropriate fake test data isn’t easy. Real world data includes errors, unexpected variations, and inconsistencies. Good development requires testing how the system reacts to such data. Thus, using existing, real-world data might be the best solution. The problem: such data could include consumer personal information, including sensitive information.  Returning to the bank metaphor, this is like storing gold in the bank while it is being constructed – it could be risky.
   
• Rapid Change. Development environments are places of experimentation. Developers rapidly and regularly swap code in and out. Sometimes they import code from outside repositories. Sometimes they repurpose legacy software components. Such code may not have been adequately validated in the existing environment. In large, complex software systems, it can be difficult to vet each individual element – particularly when multiple developers may be changing different subsystems at the same time.
   
• Transition to Production. Developed code needs, eventually, to be transitioned into production. A successful transition requires hardening the developed software by incorporating the required internal and external protections that were not in place during development. It can be difficult to catch everything, and many transitions face significant time pressure. But companies that neglect security or privacy in the rush to release a working product put consumers at risk. The FTC has brought cases against companies for poor data security practices that resulted in development shortcuts remaining in production code.

Some Tips: Software development environments have some unique characteristics. How can companies help protect their consumers while maintaining productive software development environments? Here are a few tips.

• Take seriously the security and privacy of development and test environments. Development environments differ from production environments and may need different privacy and data security solutions. But companies must consider privacy and data security in all phases of the software lifecycle. As software engineers adopt agile and other rapid, iterative development and deployment methodologies, taking a lifecycle view of privacy and security becomes even more important.
   
• If you need to use real consumer data in your development environment, remember that you must still keep your privacy promises to consumers and you must reasonably secure that data. Consider removing unnecessary data fields, restricting access, or imposing other controls.
   
• Before submitting code to open source libraries or other public venues, subject it to rigorous review. Review it for security vulnerabilities, especially hardcoded keys and logins. Not only does this protect you, your employer, and your customers, it also improves the quality of the code for the community.
   
• Use caution when abstracting out security, even temporarily. If you use security or privacy stubs for convenience while you develop features, be sure to document such use and replace any stubs with functional code before deploying. These types of abstraction, if left in production code, can be a major source of vulnerabilities.

Software development environments are important tools. Developers use these tools to update existing products and services and to build new software products and services. But such environments need to provide sufficient safeguards to ensure privacy and data security even while they foster experimentation and innovation.