A bit about each of the Services
What is it: “Fluentd is an open-source data collector, which lets you unify the data collection and consumption for better use and understanding of data.”
How we use it: We get all the data from wherever it is into a central location and ready for the next piece
What is it: “Amazon S3 is object storage built to store and retrieve any amount of data from anywhere – websites and mobile apps, corporate applications, and data from IoT sensors or devices. It is designed to deliver 99.999999999% durability, and stores data for millions of applications.”
How we use it: Sounds like a good place to put all the data, its reliable, inexpensive, and integrates well with the next step.
What is it: “With Lambda, you can run code for virtually any type of application or backend service – all with zero administration. Just upload your code and Lambda takes care of everything required to run and scale your code with high availability.”
How we use it: Ok, normalization, processing, and data prep, as well as high-level, alerting based on pattern matching. Getting awfully SIEMy up in here.
What is it: “Amazon Athena is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. Athena is serverless, so there is no infrastructure to manage, and you pay only for the queries that you run. Athena is easy to use. Simply point to your data in Amazon S3, define the schema, and start querying using standard SQL.”
How we use it: Here, we go for the bulk of our processing magic. We take the knowledge developed from years of information security experience. With enterprise and government systems and everything in between, we distill it down to SQL queries. We also are integrating information from threat feeds. Sound crazy? Maybe, but it works.
What is it: AWS Glue is a fully managed extract, transform, and load (ETL) service that makes it easy for customers to prepare and load their data for analytics.
How we use it: Provides some structure for analysis, We define our data in glue and it integrates right into our queries. Allows us to abstract the actual data format and make the data useful. If it changes, schema on read, update the schema, rerun the query, no heavy indexing required.
What is it: Amazon QuickSight is a fast, cloud-powered business analytics service that makes it easy to build visualizations, perform ad-hoc analysis, and quickly get business insights from your data. QuickSight is 1/10th of the cost of traditional BI solutions with no upfront investments, no expensive hardware to purchase or infrastructure to manage, and no additional license or maintenance fees.
How we use it: This one seems obvious, let’s visualize the data and turn millions of lines of log data into an actionable picture.
This is the high-level workflow for NGS today.
Is this the SIEM killer? We think so. We are still refining and expanding the capabilities, making things cleaner and easier to use. It’s been an amazing journey to this point, and our mission continues to be finding ways to empower smart humans to make information security work. Want to check it out?
We, here at the Mosaic 451 Cloud team, are always happy to chat. Drop us a comment here or send me an email. Micah.firstname.lastname@example.org.
Thanks for the read and more to come: My next blog is about how we integrate threat feeds and our baseline searches for security.