30 Aug ChaosDB – what it is and what are your next steps?
What is “ChaosDB”?
Late last week, the researchers at Wiz made a rather stunning revelation about Microsoft’s flagship NoSQL offering, CosmosDB which was named ChaosDB by them. In short, Wix were able to access customers’ CosmosDB primary keys using the Azure Cosmos DB built-in Jupyter Notebooks feature. Microsoft security response centre has published their initial investigation notes. This is an extremely serious security flaw obviously, as this would have given complete, unfettered access to customers’ data to an attacker. The Microsoft post indicates “that no customer data was accessed because of this vulnerability by third parties or security researchers.” based on their investigation, which means the flaw may not have been found out and exploited by bad actors based on the information available at this time.
Nearly everything we do online these days runs through applications and databases in the cloud. While leaky storage buckets get a lot of attention, database exposure is the bigger risk for most companies because each one can contain millions or even billions of sensitive records.
What do we know technically?
Microsoft introduced the Azure Cosmos DB built-in Jupyter Notebooks feature in 2019. In February 2021, this was automatically enabled on all CosmosDB accounts created from that point forward, even if not used. The notebook container allowed for a “privilege escalation” into other customer notebooks – which means that using the notebook context, the attacker could gain access to other customer’s database primary key and hence, all data! The Microsoft email to customers have said that VNET, private link, or firewall enabled CosmosDB account – which indicates that the data would probably not be accessible even it the key was leaked, though it is not clear if “Accept connections from within public Azure datacenters” was checked in – whether the firewall would allow the traffic from the Jupyter containers.
A webinar is coming up from Wix with more tech details which we recommend registering for.
Image reproduced from ChaosDB: How we hacked thousands of Azure customers’ databases | Wiz Blog
What should you do?
- As per the guidance from Microsoft, you should immediately regenerate your primary key.
- If you practice DevOps or (if you don’t mind my buzzwordiness) DevSecOps – this might have been already remediated. Credential rotation is one of the authentication best practice recommended by OWASP. This is a good reminder that all your privileged credentials should be routinely rotated and updated securely, ideally without human intervention. Unfortunately, at this point, CosmosDB doesn’t allow an Azure Key Vault based rotation, so PowerShell or similar runbooks are the best bets to the best of my knowledge.
- Network isolation with VNET or Firewall for your CosmosDB account is a good idea, where feasible. This provides an additional layer of security.
Where do we go from here?
This is a very serious development and hopefully we would see a RCA from Microsoft soon. The unfortunate reality is that this might exacerbate what we call “PaaS hesitancy”, especially from large enterprises. We have seen PaaS becoming the option of choice for many enterprises due to the numerous advantages – but not knowing the intricate details of what is enabled, what is opened and when things are done features in the top typical objections – and this confirms the validity of that line of questioning. While Microsoft has responded to this very swiftly as they were notified (which was called out by Wiz), there are some key points – very succinctly summarised by Christophe Parisel in his LinkedIn article – CosmosDB vulnerability calls for rethinking PaaS integration | LinkedIn. In particular, the opt-in experience is probably going to become a key one for enterprises. We don’t want the pace of innovation to reduce, that is why we go to the cloud in the first place, but we do want to ensure enterprises get the control that they desire – so this doesn’t become a blocker in cloud adoption and capitalising on the velocity multiplier levers that cloud delivers on. Looking forward to some positive steps in that direction and a strong response from the cloud industry with solid remediation steps across PaaS lifecycle as we navigate the aftermath of this finding.