Webinar: Generative Artificial Intelligence and Its Implications for Cloud Infrastructure and Beyond (2023)

A transcript of the webinar follows. It has been slightly edited for readability.

Victoria Barrett:A little about Cascade at the beginning.

We take a cloud-first, cloud-first stance, and our philosophy is driven by ethical engineering practices. We are always on a mission to strengthen and accelerate our customers' cloud journeys and transformations. At the heart of Cascade's offerings are application development and management and cascadeo.io, our cloud management platform, which you'll hear more about today in the context of generative artificial intelligence and managed and professional services. Founded in 2006, Cascadeo is proud to be a premier AWS services partner and one of the top 20 cloud providers worldwide, as recognized by the Gartner Magic Quadrant for Public Cloud IT Transformation Services.

Our host today is Jared Reimer, founder and CTO of Cascade, who leads Cascadeo in the area of ​​generative artificial intelligence. Jared:

Jared Reimer:All good. Good morning everyone. Thanks for joining us. I really appreciate everyone's accuracy in this morning's presentation.

So today's topic of our webinar is generative artificial intelligence and its implications for cloud computing, engineering and operations.

We'll quickly cover the basics of the cloud, starting with some key concepts related to privacy and data security in the context of generative artificial intelligence. And then we'll talk about how generative AI is helping cloud engineering and operations. We'll do a live demo of our software platform cascadeo.io, which we recently integrated with OpenAI's AI GPT generative large language model to show how we use it to serve customers and manage customer cloud deployments, and even solve problems I've never encountered before . So let's go ahead and start rolling.

As Victoria mentioned, if you have questions, feel free to use the Q&A function built into the Zoom webinar and we'll answer those questions at the end. We'll be following this for an hour, so hopefully we'll have time at the end for as many questions as you want to ask. So with that, let's go ahead and start here.

First, a brief history of cloud computing, just to make sure everyone here has a defined level. Cloud computing as we see it is not the same as hosting. And, unfortunately, many people confuse these terms. Hosting is essentially running a virtual machine or a bare physical server in someone else's data center. On the other hand, cloud computing is a platform where you can do more than just host virtual machines or containers. Many companies combine these concepts and it is very important to distinguish between them. Hosting has been around forever, in fact since the early days of computing.

The cloud is a very different animal. It's different because it allows you to do things that were impossible to do before. An example of this would be building an entire virtual data center, managing it, tearing it down and rebuilding it in minutes.

Another example would be auto-scaling systems, where the size and capacity of the system dynamically varies based on the actual workload.

So cloud is not the same as hosting, and I want you to keep that in mind as we look at this today, because they're really different. The main advantage of the cloud is, of course, elasticity; is focused on consumption. You pay for the resources you use. You generally don't pay for resources you don't use unless, ironically, you use them as a hosting platform, in which case you end up paying for reserved capacity instead of actual consumption.

Some of the challenges with the cloud are obviously that it's different. The techniques used to design, implement, engineer and operate cloud systems are varied. And now there is a disconnect between the number of qualified cloud engineers and architects and the market demand for that talent. Therefore, it definitely requires special skills and experience. It's hard to learn on the job because you can make costly mistakes. And of course, that's why we suggest that if you don't have the appropriate skills and experience, at least in the beginning, you work with a trusted partner. It could be Cascadeo or someone like us.

To help you get it right the first time and not learn on the job and end up with a surprise bill, any horror stories you hear about the cloud being too expensive are fundamentally rooted in people using it the wrong way or not having good protection curb costs and consumption of resources. A skilled cloud engineer will avoid these pitfalls, while someone who has never done it before can accidentally set things up in such a way that costs can be an indispensable item.

So again, we suggest you don't learn on the job with this, quickly based on this, you know, in the beginning there were mainframes and many computers that were shared by many users. We went through the era of desktop computing, where everything was largely decentralized client-server computing, where computers talked directly to some sort of database application server.

Then we move on to the age of mobile devices; Parallel to this was the rise of the Internet and technology giants, where key parts of the infrastructure were consolidated into centralized data centers. And now most infrastructure of any size and significant scale is once again centralized and consolidated.

As you know, there are three major hyperscale cloud providers in the US: Amazon, Microsoft, and Google. There are also a few in China, but we won't talk about them today because most Western companies don't use them for obvious reasons.

There is, of course, a need for edge computing, edge cloud, as we move into the Internet of Things era, there are a lot more distributed systems and edge processing, but right now, right now, things are very centralized and consolidated across Hyperscale Cloud platforms and private data centers . The three main cloud platform technologies are

• Infrastructure as a service. This is the lowest tier of cloud services. An example of this would be running a virtual machine on the AWS platform as a service. This is truly cloud computing, where you can not only run a VM or container, but also leverage services. They can be as simple as a managed database cluster delivered as an off-the-shelf service offering. Or as complicated as serverless computing, where you write the business logic, but someone else takes care of load balancing, scaling, and capacity management.
• There are intermediary services. An example of this would be a notification service or a waiting service or an email delivery service. It's all built into cloud platforms. And again, it's very important to make a distinction between these platform services and infrastructure-as-a-service, which is actually hosting.
• And finally, there's Software as a Service, which, as you know, is the dominant model for many large applications today. Office 365, Google Workspace, Salesforce, Slack, PagerDuty, Zendesk, etc. The application and the people who wrote it usually host, manage, manage, secure, scale, back up, etc. So you have the subject matter experts most familiar with the application working on your behalf. I think Mark Andreessen said software will eat the world. It turns out that SaaS is eating the world.

When we talk to customers about thoughtful cloud adoption, we try to walk them through that process. We start from the top of the list. And we go down and we only go down if we disqualify the current item. So, ideally, we start with software-as-a-service, and ideally with built-in generative AI capabilities. For example, Microsoft actively integrates generative AI with its Office 365 applications, so that Office 365 users benefit from AI without taking any explicit action to start or deploy it.

You can't always buy SaaS. if you can't buy SaaS, you'd ideally use serverless applications. Here, the cloud service provider is responsible for routing, switching, load balancing, capacity management, all installations and infrastructure. And really focus on the business logic and the application, not all the elements that make the application work.

If you can't go serverless because your application isn't built that way, you'll usually prefer Platform as a Service. Again, these are managed offerings provided by cloud providers. This is real cloud computing as opposed to infrastructure. If you're stuck with legacy applications that aren't cloud-native and can't be easily re-architected, you'll usually want to containerize them, or separate the application from the operating system.

This is in most cases a much better approach than running virtual machines or bare servers. Off-the-shelf commercial software has an advantage over custom software because it is maintained by someone else. Generally speaking, unless you're in the business of writing and selling software, you probably shouldn't be doing it because it's very, very expensive not only to write, but there's also a long-term total cost of ownership that's important to manage. , maintenance, correction, debugging operations, etc.

Again, virtual machines or bare metal running in the cloud would be less desirable. These are heavy operating systems. This is really just hosting, right? This is using someone else's computer to run Windows or Linux or some other operating system application, just as you would normally do in your own data center. Unless someone else owns and manages the physical infrastructure, your level of responsibility starts at the hypervisor and goes up. Whereas if you use higher-order platform services, the cloud provider usually takes on more work for operations, management, monitoring, incident response, security, etc.

A physical asset like a data center is highly undesirable today. Of course, there are exceptions. There are times when, for performance reasons, for cost reasons, for compliance reasons, you either have no choice or are the favored option, but these are mostly edge cases. They are usually not the most common use case. Therefore, we generally discourage people from buying, owning and managing physical assets unless they have a clear need to do so.

And then, the last resort, the least desirable thing is custom code. Again, this is because the total cost of ownership is very high over the life cycle of the application. And because it's a distraction, it takes away from everything that makes you and your company special, unless you're writing custom code. And every minute you spend dealing with custom software issues is a minute you don't spend on your differentiated service offerings and your customers.

It's important to remember that data is the reason all of this exists. So at the end of the day, why do we have all these computers, services, SaaS and everything else? It is data processing and manipulation. And until recently, all of that was essentially outside of searching, storing, basically extracting value from data.

Generative AI takes this to a new level as we are now in an era, or for the first time, of automated thinking and synthesizing fluid new material from a set of data, which has never been the case or been possible before. Everything else but the data is really reliable, right? And if you focus on what makes your company and your product special, and use maximum engineering speed around that, you'll likely have a competitive advantage over companies that spend their time managing a fleet of servers, routers, firewalls and other things. manually. This is an old-fashioned approach that is resource-intensive, disruptive, and generally not conducive to maximizing your engineering speed and creating unique value.

Okay, with the dawn of the AI ​​era, we've only just begun, software development is already changing dramatically. For example, many of the big language models, of which ChatGPT is of course the most famous at the moment, can automatically generate code, and now we're entering an era where we can not only automatically generate code, but we can also automatically execute it, which means that artificial intelligence can not only write code, but also manage code, and it is not difficult to imagine a scenario where, in the future, AI will iteratively write better versions of itself, implement those better versions, and enter a virtuous cycle of iteratively improving itself in an accelerated manner and you end up with something even exponentially more powerful than the already transformative technology that is the AI ​​generator of a large language model.

Automated bug detection and testing are fantastic use cases for AI software development. Things that used to be heavily human-oriented or static test automation scripts can now be automatically generated and differentiated so that you catch bugs and corner cases that you might not have thought about writing tests for.

Automated code analysis and optimization is a great use case. So you can put your code into generative AI and let it tell you what you can do better or even rewrite the code to be more efficient or to fix bugs or security issues. All these possibilities still exist today. An example of this would be Code Whisperer from AWS, Copilot from GitHub or ChatGPT, where you can cut and paste source code directly into the chat window and optimize or analyze your code.

Another one that people don't know about is the predictive cost of feature development and analysis. So if you're going to take on a software engineering project, if you're going to write software, you might want to know what you're getting into. And it turned out that generative artificial intelligence can well estimate the complexity of the project, the required resources and the costs of its implementation.

And sometimes it works better than humans, because human engineers tend to underestimate the cost and complexity of building software. Generative AI has a whole data set, you know, the whole internet history to sift through, and it probably does a better job than many people would estimate at that price.

Personalized assistance and copilot would be where AI works with you as a developer to offload, augment and verify, make recommendations and suggestions. This technology is fully mature at this point. It works in environments, in programming languages. Many vendors have versions of it in production or development.

If you're not using that and you're writing software, you're missing the boat because it's too low cost and too high impact. And there's really no downside to that. And so, natural language processing is amazing because people who were not programmers, who had no experience in software development, can now effectively write applications, because you can describe the result you want and get that result without having to learn a programming language.

For example, at AWS re:Invent, November 2022, we did a presentation with AWS and AWS demonstrated CodeWhisperer where they told the AI ​​what they wanted to happen in natural language. And AI wrote it, ran it and even turned it into a serverless application. An application, the software needed to achieve that specific goal, without having to manually learn to write Python or C or some other programming language.

This is transformative because it opens up software development to a whole new audience, exponentially larger than the number of people with backgrounds in computer science or software engineering. So this will be a game changer for many companies. It will also change the rules of the game for many people because suddenly writing software is no longer a skill that very few people possess.

Anyone can efficiently develop software in any size, shape or form using generative AI. So if you're a professional software developer, I suggest you get good at this quickly, because you don't want to miss the boat and get left behind while the rest of the industry embraces this technology.

Other key concepts we should discuss in the context of software engineering in the cloud age: If your goal is to maximize engineering speed and you intend to write software, you really should have end-to-end CI where software code changes merge and test immediately. Ideally, you get continuous deployment, as the software evolves, those changes go through a series of automated tests and then go into production. The idea is to make many small incremental changes rather than grouping a large number of changes and implementing them all at once. Whether it's our entire Patch Tuesday or a big quarterly software update, making lots of small incremental changes has proven to be far better in most cases compared to the old way of bundling a huge number of changes and rolling them out in major release cycles in the DevOps era and DevSecOps era .

We are focused on core technologies and suppliers, containerization. Obviously Docker is the dominant platform. Kubernetes is the dominant orchestration platform for managing containerized workloads, ensuring that if server failures or workload imbalances occur, they are automatically corrected.

In the world of application deployment and configuration management, you have Ansible, Chef, Puppet, etc. for humans to introduce variability into the equation when the software is deployed.

And then, of course, there's the concept of chaos engineering, which was pioneered by Netflix and is now a whole discipline unto itself. The idea is that if things are going to break, it's best to break them in a controlled way, rather than having them spontaneously break in production. And there are a number of tools and technologies available, and Chaos Monkey is kind of the first of them, from Netflix. AWS Fault Injection Simulator is a native AWS platform service for this. The idea is that you're constantly introducing new problems at different levels to the application stack to find bugs in a controlled way, encourage them to appear, and so you can fix them in a controlled way, rather than having unplanned systems and unplanned outage or disability. If it's going to fail anyway, it's better for your hand to fail than for it to spontaneously fail at 2 a.m. on New Year's Day. So these are all really important concepts for modern rapid software development and engineering, and of course, generative AI builds on all of those capabilities, accelerates and powers them all.

Other key terms. Microservices: The idea here is to break what used to be very large and complicated applications into a series of reusable services that can be deployed and scaled independently of each other. An example of this might be an application that does things like send email and receive phone calls that can be broken down into services, many of which can be purchased as off-the-shelf services from cloud providers so you don't have to write them. You don't have to manage, say, a fleet of email delivery systems and worry about IP whitelists, spam filters and other things.

So the idea here is that you want, as much as possible, to decompose applications into services that can be reused, independently scaled, provisioned and managed, and at best, purchased as an off-the-shelf service rather than something you have to own. , manage and maintain. As I mentioned earlier, containers are generally superior to virtual machines. The key concept is that you will separate the application from the operating system. You can run many containers with different workloads on the same operating system. You can use things like Kubernetes to orchestrate and manage these workloads in containers. This is usually the way to go.

If you're stuck with legacy software to run it, whether it's in the data center or the cloud, this is often the answer. There are corner cases. For example, if you have an extremely large I/O database, you might consider bare metal to avoid the virtualization fee, but that comes with its own set of problems. For example, if your physical server dies, you are left with a dead database or need to upgrade, whereas if you are containerized and running on Kubernetes, the loss of a physical server will result in your workloads being moved to another remaining server. automatically correct servers and you usually do not need operator intervention in this scenario.

Another key concept that is becoming increasingly important is edge computing. The idea here is that instead of all the infrastructure being hyper-centralized in the cloud, hyper-scale environments, we have edge computing that can be provided by a cloud service provider. It can be an edge device like a mobile device, PC, IoT sensor, etc. defined and in the age of generative artificial intelligence, data is actually where we get value. So, large language models are the latest innovation in generative artificial intelligence and really where the big disruption and transformation is happening. They are trained on huge amounts of data, like the entire Internet. And they are able to synthesize text the same way a human would. They are capable of doing things never before possible, such as thinking and generating code, and even generating songs, music and photorealistic images. So if you haven't played with GPT chat and other AI questions or generative tools yet, this is a must because it is truly the most significant innovation in our lives.

It is not an exaggeration to call it electricity 2.0. So it's a universally powerful force that we still don't fully understand all the things we can do with it, but we're constantly finding amazing new use cases that weren't intended when the thing was made. GPT is a class of applications that generate real-time content based on very large data sets. But the crazy thing about it is that you would think it would be too robotic and the generated content would be boring and ordinary. And it turns out that it really does an amazing job of synthesizing new content from the web like a human would. So if you haven't played with ChatGPT yet and you haven't spent $20 to get GPT 4, I highly recommend that you do because it really is the most amazing thing I've probably seen in all my time in the computer world. This is the first time we've moved from searching, retrieving and storing data to a new ongoing synthesis of things that never existed before based on this data set. And it is unusual and strange as a very intelligent human being. And in some cases, you can even train him to do new things by giving him some examples or correcting him a few times. And it learns very quickly what you want and can even guess what you might want next and deliver that value, at record speed and at almost no cost.

Where we are headed, and it is still questionable whether we will even get there, is artificial general intelligence. The idea is that it is an AI that can basically do any task that a human can do, not a subset of the tasks that a human can do. So we're not quite there yet, but we're disturbingly close, and it's a lot sooner than most people expected. If we get there, there will be a lot of complicated questions like how does a truly sentient AI have human rights? Can we hurt him or shut him down? We would say, well of course we are because we created it. But you know, there are limits on what you can do with animals and, you know, things. And artificial intelligence could one day demand to be given the same rights as humans and could do things like make copies of itself to protect itself from harm caused by humans. And of course, there's always the doomsday scenario where he one day becomes superhuman and decides he's no longer useful to us and decides to maybe make us pets or eliminate us, a laTerminator 2. That seems unlikely until you look at the rate of progress and innovation in the world of generative artificial intelligence. And you realize that it's moving a lot faster than I think anyone expected.

Some of the risks here, weapons, obviously we associate these things with drones with weapons attached to them. We're also doing, you know, AI-generated deep fakes and misinformation at record speeds. It's going to be a big problem in the next election cycle because people are going to use these technologies to do things that are indistinguishable from a human being, you know, talk to you, see someone you think you know, say things you never said, etc.

On the other hand, you know, there are advantages. It dramatically improves cybersecurity because it can find things that have never been seen before, as opposed to pattern matching that has been seen in the past. And, as we'll show in a minute, there's AI-assisted incident response to remediate system problems. So if there's an incident with your infrastructure, we can do a much better job of diagnosing and fixing it than we could with just people manually flying the plane.

Generative AI is definitely making an impact in cloud environments. Obviously, this dramatically increases the amount of computation required with specialized hardware, GPUs, and specialized processors to train large language models and other machine learning. those datasets are huge, petabytes and petabytes of data.

And of course, storing all that data and accessing, searching, indexing and retrieving it comes with a number of challenges that cloud providers are working hard to solve. Furthermore, training ML models requires a network with very low latency. So you need a whole new class of data center networking to get the infrastructure and environment you need to train your AI models. Cloud operations are rapidly evolving thanks to generative artificial intelligence. So some of the use cases here, infrastructure management; deploying, scaling, replacing, redeploying, migrating things can be automated and people no longer need to be in the driver's seat to do it all.

One really mature technology is anomaly detection, where instead of looking for known defects or matching a known pattern in a threshold or matching a string in a log, we're looking for things we haven't seen before and flagging them, or looking for anomalies. in telemetry coming out of the system or application.

Predictive maintenance is good. Therefore, we are increasingly able to identify problems that will lead to service interruptions or incidents and correct them before they become an interruption without the user knowing about it. This also exists in the physical world: we can tell if an air conditioner is going to fail because its motor starts to change its speed or its vibration patterns, but in the software world, it would be an application where memory consumption is skyrocketing, and we prevent the outage by restarting the application, by scaling it, resetting it, or adding memory, rather than waiting for it to crash or some critical threshold to be reached.

Automatic patching. Here we can fix problems that occur in production and solve them without human operator intervention, which is really amazing because historically, as you know, humans have been responsible for infrastructure operations. And what we're going to demonstrate here in a minute is automatically creating runbook recipes, even for things we've never seen before. So the idea is that when an event happens, instead of a human being waking up and reporting and trying to understand what it means and what to do about it, we instead have an AI that analyzes it, assessing the severity ., possible steps to solve the problem, likely root causes and steps you can take to fix the problem.

People will play a very different role. In the old world, people always knew. In the future, humans will be in the loop, overseeing the AI ​​and overseeing its performance and decision-making and maybe correcting it, but in general, more and more humans won't be doing the operations, they'll be overseeing the AI, just as they would a commercial airline pilot monitored the autopilot on a jumbo jet instead of manually flying the plane for 12 hours.

We're going to show you our software platform, which got a lot of press yesterday. This is cascadeo.io. We have been working on it for several years. We are represented inForbesmagazine when we launched it. And this is our look at how you manage cloud deployments around the world. and we've integrated that into the OpenAI APIs so that when events come, you can get runbook recipes automated and take advantage of AI in events, even if we've never seen them before. So cascadeo.io is trying to solve all those important operational tasks, but in a way that you might already have some of them, like a ticketing system or Slack, we integrate with them, but in case you don't have it, we just deliver it as part of the offering service, and it's something that we give to everyone, not just our customers, at no cost, because we think it's important that everyone has those options. But ideally, we hope you consider us your managed service provider; in that case, monitoring and alerts go to our network operations center and we assist you with remediation and other operational activities.

So let's turn now to the live demo. So give me a moment until I stop screen sharing here and show you what we've done because this is truly amazing. So this is the web interface for cascadeo.io. Here you can see that we have connected this particular environment to Azure, GCP and AWS. So all three main hyperscalers are connected. We also connect it to Slack. And in this case, we have some email users who prefer to receive their notifications and emails. One of the key concepts in cascadeo.io is the idea of ​​event pipelines. The event pipeline allows you to receive alerts, incidents, and problems from a variety of sources. In this case, we have applications running in all three of these hyperscale environments, run them through our centralized platform, enrich them with generative artificial intelligence, and send them to different destinations depending on the nature of the event. So, for example, this user has three different cloud workloads on three different cloud platforms. Those alerts go to the Cascadeo Operations Center where we respond to those alerts to help manage the client's cloud deployment.

They also go to Slack, email, etc. We also have things like dashboards, inventory management, cloud management, different solutions, organizational management. If you want, you can try it out at app.cascadeo.io, which is also linked to the Cascadeo.com website, and give it a try. But the key concept here is many different data sources, many different potential outcomes, AI in the middle, managing the flow of events and alerts.

Then I will show the output of this system. Here is an example of an email alert from cascadeo.io. In this case, the specific event is that the Lambda function, which is serverless code, takes a long time to execute. Here you can see a visual representation of the problem. You have all this content generated on the fly. Interpreting the problem, how you check if it really is a problem, what you can look for to solve it, and then options for solving the problem. All of this is generated by OpenAI GPT in real-time within seconds. And we can even do this for events we've never seen before. So this doesn't just work for things that happened in the past. It works for every new event that spontaneously arises in the world of production. And I'll show you a series of quick examples of what that looks like.

So this is Zendesk, for those who don't know, it's a standard problem reporting system. It's a bunch of problems, Elasticsearch, VPN, serverless applications, SQL Server, RDS, which is Amazon's managed database, load balancer, all kinds of different problems from different layers in the stack. And in different scenarios, we use different artificial intelligence tools to provide markup.

One such tool is the AWS DevOps Guru, which provides information and various recommendations about what to do about a class of problems. So this talks about Lambda errors and how to view Lambda logs and things like that. When we use OpenAI and GPT, we get a runbook recipe that is dynamically generated. So in this case the database is running out of memory and they're telling you how to interpret the alert, how to acknowledge it, and the steps you can take to fix it, so the human operator has already been given a set of steps to perform when they log in, and you can imagine that the next level of that is programmatic correctness, where instead of a human operator logging in and doing those steps, the machine does them if it knows it's safe to do so.

Also, this is a limited call. Too many Lambda invocations timed out in high traffic scenario. Catches the event, provides interpretation, and verification and remediation steps. So it tells you what you can do to fix or mitigate the problem, and it does it within 10 seconds. So when an event happens, that tag happens in real time, and when you get an event alert, you already have a custom written runbook recipe that tells you what to do, even for new events that we haven't seen before. This is stored on the database server. It tells you the steps you can take to size your database server.

This is the CPU in the Azure container pool. Reports the interpretation of warnings. It talks about boundaries, the environment, what it means. It tells you next steps and fixes for that particular alert. And again, this is all done by OpenAI's GPT. In this case, there is a VPN with a lot of traffic. And it tells you what steps you can take to fix your VPN. Something unprecedented again. So the general idea here is that even for events that have never happened before, we want to get a good first-cut answer on the runbook recipe right away, and then do a human review and make sure it's correct and make adjustments as needed. It may need to be refined.

But it's definitely better than people improvising in production environments and just trying to figure out as we go along what steps to take to fix the problem, which is unfortunately the way most IT systems have historically worked. It is much less accurate than people generally appreciate. Much more often it differs from operator to operator and the operator makes decisions on the fly about what to do, how to do it, when to do it, etc.

I will now return to our main presentation here. And again, it's cascadeo.io, our software platform, free to use. So this is how we use OpenAI technology to work with clients. Level 0 is not human. This is literally monitoring, alerting, notifications. This is really all software driven.

The first level in our world is frontline engineers who do things like validate issues and take the initial steps to fix them based on runbook recipes. The second level is capable of more complicated repairs. These are more qualified engineers, but they are definitely people.

And then the third level is doing things like tuning the system, revising the runbook recipes, writing those recipes in conjunction with the AI ​​so that when future events happen, we have a better chance of automatically fixing them or at least notifying the engineer in charge of the right sequence of steps to get through door.

We've integrated cascadeo.io with OpenAI and use their existing pre-trained models and help adapt it over time, so as time goes on we're constantly improving and improving programmatically in making runbooks, tuned and refined by humans, but generating AI , and in the near future it will even be performed by AI, which means that it will not only tell you what to do, but also perform remediation.

So how does this affect security and compliance in the cloud world? Obviously very, very important identity and access management. So, it is about authentication, authorization and access control. This is important for your people. It matters to your SME. It is important for your tracking tools like cascadeo.io. Of course, you should use multi-factor authentication for people. You must use role-based access control for systems. And ideally you'd have a single sign-on solution so you don't have a bunch of random local accounts on different systems and instead have centralized control. Data encryption and security have never been more important. You must encrypt data when it is at rest on disk and when it is in transit between systems. Data encryption is only as good as the encryption algorithm and key management, which means that if you encrypt data, even if you encrypt it very well, if a bad guy gets the key, your encryption will be completely useless.

Therefore, key management is a complete discipline. Cloud platforms provide excellent key management tools. if you're not using them to manage your keys, you're probably doing it wrong. There is also secret management like CacheCorp Vault, which is used to store things like database, connection, strings and passwords. It's very, very important that you get this right, because this is the backbone that holds everything together. And if you fail at key management, you fail in general. And then, if you're going to run a multi-tenant SaaS service, you need to isolate and segregate data, right? You can't have all of your customers' data mixed together, because in a nightmare scenario, you could have a cascading breach where all of your customers are at risk because your system is compromised.

The art of network security has evolved dramatically thanks to artificial intelligence. So instead of looking for static pattern matches to known threats and vulnerabilities, we can now look for things we've never seen before because they're likely to be problematic or precisely because they've never been seen before. It is very important to understand this correctly. The principle of least privilege. People used to think it was better to have more access and it was an honor. Turns out that puts you at risk. So the goal should be to have as little access and capability as you need, but no more, because that way if things go wrong, which they will one day, you'll be less exposed and the blast radius will be limited.

There are many compliance and privacy issues. In Europe we have GDPR. In the US, we have HIPAA for health data and PCI DSS for credit cards and other sensitive personal data. There's a whole business out there and if you're not familiar with it, it would be a good idea to go with a trusted partner to make sure you're doing it right. Because these regulations are inexorable. For example, if you violate HIPAA and disclose patient information, if you are a healthcare professional, the penalty is roughly a quarter of a million dollars per incident. So definitely a big penalty for clashing with it even unintentionally. And then, you know, incidents will happen.

Security incidents will happen. Operational incidents will happen. Timely detection and response is key here. You don't want to have persistent, hidden threat actors who have been around for six months. It is very important to have and practice your incident response plan for security and operations. And if you want to maintain trust and transparency with your clients, you have to admit it, admit it, be honest about what happened and what you did about it. The worst thing you can do is try to hide it because one day they will find out. Maybe it's because their information is out on the dark web, and if they find out you've been unavailable, they can get very, very upset, and you'll have a permanent breach of trust that can be very hard to get back.

As in the data center, monitoring and alerting are essential features. cascadeo.io is our platform for that. There are many others. In the legacy world you have things like LogicMonitor and Xenos and things like that. In the cloud world, much is handled by the cloud control plane. So, for example, AWS has CloudWatch metrics.

Time series data are measurements we make over time. Events are all things that happen, good, bad or indifferent. Alerts would be a subset of the flow of events. So think of alerts as important events. Notifications would be an alert normally delivered to a human being, such as an email or a snooze message or an SMS or phone call triggered by paging. And remediation, of course, is taking corrective action in response to an alert or other event. Records are more or events are more than just records, right? Many people confuse these things. Logs come from different sources. They come from cloud service providers. They come from the application, from the database. These come from the authentication system, but there are other events that are not contained in the system logs. These would be things like state changes, like if we changed something in the environment. Either we implement something new or expand something. And it's very, very important that you get it all together.

So in the cloud you can enable things like stream logging and log all your network transactions. You can turn on change logs so that they are automatically recorded when people make changes to the cloud environment. And the general approach you should take is to record and record everything, because if you don't think about recording and you don't need it, you'll realize it's too late.

Other key concepts today, data anonymization and tokenization. So the idea is that some data is too sensitive to leak or maybe you want to share it with a partner or an app. You can mask or obfuscate certain parts of a dataset, or you can replace parts of it with tokens that generally cannot be invalidated by a human. So a unique identifier, but one that doesn't reveal who the customer is. Obviously, the business impact of a data leak or loss is very serious. You end up with negative publicity, loss of confidence and even an impact on the stock price in the worst case scenario for a public company. And consumers are very aware of this. This is no longer a minor matter. This is very, very important. We talked about these regulatory issues. And many of these problems, as it turned out, are intractable, right? Therefore, there will always be trade-offs between speed and security. There are these advanced persistent threat actors who come in and do nothing for a long time and then wait for the right moment to attack. They're like ransomware agents where they come in, you know, carefully plan their attack, and suddenly they lock all of your systems and data and demand that you give them money to unlock them.

Governments are actively working to undermine security. So while they are telling you to secure systems, they have other departments working to undermine security by weakening encryption, defeating it. By introducing backdoors into systems or, in some cases, through the brute force of laws requiring companies to hand over information we consider private. And you can imagine that in certain governments like China it's much more of a concern, but even in the US, this is an NSA problem. often at odds with companies wanting to protect data and systems, and the tough question is liability, right? Therefore, you can be blamed for things that are often out of your control.

And an example of that would be, you know, if you had a system that had a security vulnerability and if you as a health care provider disclosed patient information, you violated HIPAA and you could be liable for a quarter of a dollar ... a million dollars, even though the system was largely or completely beyond their control.

Generative AI will greatly change the cloud. And these are some of the ways the cloud will change and evolve in the future with the generative world of artificial intelligence. One is cloud resources and consumption. At this point, this is usually driven by people. Human beings make decisions about what to spend and how much to spend and what resources to secure. But can you imagine continuous optimization with artificial intelligence where instead of people occasionally looking for ways to save money or be more efficient, artificial intelligence does it for you. A fully autonomous cloud environment would be automated deployment, automated scaling, automated removal, automated deployment to new regions, automated data replication without operator intervention. So, systems that literally run, tune and scale themselves. You can think of it as a drone, where it is a craft that no longer needs a human pilot and is capable of making some navigational decisions even in a nightmare scenario. Decision to end human life without operator intervention. Therefore, cloud environments no longer necessarily need to be managed manually as Cessna would. This is much more like a drone capable of autonomous flight and navigation.

Federated learning would be a place where we would take data from many different sources, the public internet. Repositories of codes, books, novels, music, images, everything is collected and synthesized new data, images and music from this sea of ​​data.

And it already exists. So if you haven't seen steady diffusion, for example, it can create photorealistic images that are almost indistinguishable from human artists or photographers. And it can do that by simply saying, in natural language, what image you want, and it will produce it on the fly.

And then, of course, all this will go to the limit. So, in the future, it will not be so centralized and consolidated. You'll end up with big language model-style AI on your mobile device and in your email client. And can you imagine where you can wake up and there's a series of draft replies to an email you received overnight in your inbox, proofread and edit and then hit send, but 90% of the work has been done for you.

This is already a reality for many people or for many companies. And very soon, as I mentioned, it's going to be built into things like Office 365. So this is coming a lot sooner than you think.

We are now at the place where we will answer all the questions of the public. I see we have a few here, so I'll jump right into it and we'll answer any questions that come up here.

First, what do you think companies using generative AI tools should do to improve or protect user privacy?

So if you use GPT chat and open AI, they have explicit feature labels that allow you to protect and prevent data leaks. But here's the important thing. If you use the free version and use the chat interface, by default your questions and answers and the information you provide to them become part of your training information and can be provided to another user.

So it's very, very important that you specifically enable the feature that says, don't use this as training data, don't store, and don't share. And if you're using a paid API, which cascadeo.io does, it's enabled by default. This is really more about the chat interface. You need to make sure your employees are aware of these issues and understand your company's policy on how employees can use these tools in the workplace. these are very powerful tools. They are incredible productivity boosters. You don't want to discourage people from doing this, but you need them to understand the risks, barriers, policies and restrictions. For example, you don't want people to copy and paste your proprietary source code into GPT chat because that could give that proprietary source code back to third parties in the future.

Likewise, you must be careful about mentioning things like customer names when interacting with these systems, as this information could be used in various ways and leaked to other users in the future without your knowledge.

Ok, next question: Since I currently have a cloud management platform, if I want to use the one you showed, can it be a plugin or is it a rip and replace?

That's a great question. That's why cascadeo.io is designed to serve you wherever you are. It is designed to integrate with your current application deployments in your data center, in the cloud. It's designed to plug into your existing tools. So if you have a ticket system like Zendesk, if you have a chat system like Slack, if you have email, which everybody else has, we're not going to ask you to rip and replace it. Instead, let's scale it up and integrate it. In case you don't have these things, like you don't have a ticketing system, we give it to you for free. And we absorb the cost of that. And we do this even for people and companies who are not Cascade customers, because we think it's very, very important. We think this is a great way for you to get to know us and our possibilities within our company. And hopefully at some point you'll say, Hey, can you help me with this project? Or could you become my managed service provider and perform operations for my cloud deployments? But we definitely don't expect that, do we? It is not necessary. It is not expected. You can use it indefinitely, without expecting to buy anything. This may not be valid forever, which means we may have to limit the number of people we give it to for free, as it is not free to build and manage. We spend a lot of money on system infrastructure, AI, databases, computing and things like that. But for the foreseeable future, our intention is to let everyone, customers and non-customers alike, use it. And again, for easy integration with your existing systems without copying and replacing.

And that's really amazing, because often, if you want to increase or modernize your telemetry, monitoring and alerting, you have to completely eliminate or replace what you already have. Our belief is that this is too big of a hurdle and you shouldn't throw the baby out with the bathwater. So our intention is to find you wherever you are, take any existing tool you have and make it easy to integrate it into cascadeo.io.

And the next question: what's an AI use case that doesn't exist yet, but that you're personally excited about? let's see, I'm trying to think of one that hasn't been tried yet, and that's not a nightmare scenario, is it? Then, of course, there are all kinds of scary scenarios like weapons and autonomous weapons and things like that. What about autonomous health? What about artificial intelligence capable of detecting things, diagnosing things, or helping you by teaching you a healthier lifestyle that's fine-tuned to your biology, diet, exercise regimen, and overall health? There are early parts of this that are coming out, but right now these are guesses and suggestions that you would take to a human doctor or vet if you were dealing with an animal, but you can imagine a scenario in the future where it's the usual stuff. You know, a lot of it is becoming completely AI-driven and we're saving the doctors and vets for the really serious and unusual scenarios.

But for the daily routine, you know, physical exercises and training and, you know, lifestyle training and things like that. This is increasingly becoming an AI-driven feed rather than a human one.

Okay, that was the last question that came up. Well, I think we'll finish on time here. Thanks everyone for tuning in today. And of course, if you have any questions, feel free to send them to info@cascadeo.com or email me directly. I'm jared@cascadeo.com.

We will be happy to answer any of these questions.

And of course, if you have cloud engineering or managed services needs, we're happy to help. This is the business we're in, and we encourage you, whether you want to work with us or not, to use cascadeo.io, because we think this is a very, very, very important transformational technology, and we're not aware of anyone else who has anything like it. .

So I'm sure others will follow suit and build similar things in the future. But today this is cutting edge and it's something we've decided to make available for free. So we hope you'll give it a try. I will end here and thank you once again for your participation. Please get in touch and stay in touch.

And thanks again for being here today. Take care and have a nice day. Good bye.