Automated compliance reporting
Development of a continuous compliance solution with weekly management reporting and dashboard capabilities for daily cloud operations.
ABOUT THE COMPANY
The customer is an internationally active insurance group with almost 30,000 employees worldwide and more than 10,000 full-time agents. With insurance premiums in the tens of billions, the company is one of the largest primary insurers in Germany and Europe.
The customer has an enterprise scale environment with over 130 AWS accounts and 60 productive applications which are operated by agile DevOps teams. Nevertheless, the customer needed a way to report compliance of their AWS accounts against a set of self-imposed guidelines or guardrails. The guardrails were a combination of AWS best practices, CIS benchmark rules and self-imposed rules from a customer-given policy. Our customer wanted to keep the operation teams agile but still wanted to make sure that everybody is holding up to the guardrails. There should be created a weekly report to the security council to ensure that projects fulfill the documented compliance rules. One example was a strong focus on the explicit usage of the German AWS region for storing and processing data. Other rules were based around encryption, least privilege and a stable IAM solution that keeps track of active users and credential rotation.
OUR PROPOSED SOLUTION
We developed a lambda step function that checks all of the customers’ AWS accounts against specific compliance rules (>30 rules). This step function scales with the number of AWS accounts since we use AWS organizations to check back the list of AWS accounts the customer currently owns. The solution eliminates manual steps completely. New and deactivated accounts are recognized without manual intervention.
The following examples of compliance checks were implemented:
- No resources outside a specific AWS region
- Only encrypted volumes are used and deployed
- Only encrypted RDS databases are used and deployed
- No exposed RDS databases are deployed
- No Security Groups with exposed management ports to the public internet are set up
- No S3 buckets without enforced encryption are deployed
- No active peering or VPN connections to accounts that are not part of the organization (all should be loosely coupled via public endpoints)
- No IAM users outside the central IAM AWS account
- No old access keys
- No inactive users
- No load balancers with not encrypted endpoints (e.g. HTTP)
These checks are done on an hourly basis by the lambda step function, encrypted and streamed to a central S3 bucket in a separate AWS account for further streaming to one of the customers’ on-prem warehouses. No manual steps are needed in this process. Additionally, all files that are stored in the bucket are streamed via a lambda function to AWS ElasticSearch for analysis and for usage within the cloud operations teams’ daily operations (part of the local Cloud Competence Center – CCC).
The dashboard is used daily to report compliance and to check back if all projects fulfill all customer requirements regarding compliance and security. It contains all mentioned compliance results. CloudTrail log files are streamed via a lambda function to the ElasticSearch cluster as well. This helps the cloud operations team to cross-check issues between compliance violations and the API log (CloudTrail).
AWS AS A CENTRAL PART OF THE SOLUTION
The solution that met the customers’ requirements is fully based on AWS services and was implemented without any IaaS components. AWS Config rules, which were often used by us, were not a good fit for this customer, since they only give a very decentralized view on such a huge enterprise environment (AWS Config rules per account without central dashboard or alarming). A self-developed solution based on multiple AWS Lambda functions and ElasticSearch gave us the solution and outcome that was wanted from the customer with a very low price point. The solution added maximum transparency to the huge enterprise environment.
REALIZED RESULTS AND POTENTIALS
All the customers’ security division documented compliance checks were implemented and monitored continuously. A weekly reporting was added in the process. Without user interaction, all checks are streamed to a central dashboard (AWS ElasticSearch) for daily compliance checks. All parts of the solution are fully managed by AWS (Lambda, S3, ElasticSearch) and fully automated. New accounts for new applications are added automatically to the compliance logic without manual interaction. Insights to the application environments, which are managed by smaller DevOps teams, are given from minute one without adding any overhead of barriers for the project teams.