Introduction
MSC’s core business relies heavily on data that describes every single event related to shipping, vessels, and containers. This data needs to be shared quickly and efficiently with other systems that have a similar near-live time flow requirement. Consequently, the speed of processing and relative throughput is key in evaluating overall system performance. In order to achieve the necessary level of speed and efficiency, MSC has started to integrate the Actor Model pattern into its owned and operated applications.
This integration will allow MSC to take advantage of the Actor Model’s benefits, which include improved performance and scalability. Additionally, the Actor Model’s message-passing paradigm will allow MSC to more easily share data between its various systems. By implementing the Actor Model, MSC will be able to provide its customers with a more efficient and reliable service.
MSC is committed to providing the best possible service to its customers. By using the Actor Model, MSC can ensure that its data is processed quickly and efficiently, and that its systems are able to handle a high volume of traffic. This helps to create a better user experience and to improve the overall quality of the service that MSC provides.
Scaling with the ever-growing amount of data
How did MSC arrive at the development of an actor model-centric framework? To understand this, we have to better explore MSC’s goals and the problems they faced when developing and scaling their infrastructure.
As described in the introduction, MSC has to process an ever-increasing amount of data, with strong constraints on the speed of processing and on the ordering of these messages. Initially, they developed a monolith solution to handle data and messages, based on a database-centric approach. While closer to classical models of data handling, this approach could not satisfy the time-critical constraints we mentioned above. Therefore, it was imperative for MSC to change their strategy, and move from a monolith to an infrastructure based on independent processes.
MSC is developing a novel approach to handling messages and data. They went from a monolith, database-centric approach, that was too slow and could not satisfy time-critical constraints, to a more distributed approach, based on the actor model. The new framework is designed to process large amounts of data and messages quickly and efficiently. It is based on the actor model, which is a way of structuring code that is optimized for concurrency and parallelism. This will allow MSC to process data and messages much faster than before, and to meet the time-critical constraints that are required for many applications.
Developing the Digital Factory Framework
In particular, MSC is developing a Digital Factory framework to handle the huge amount of data they have to process using the actor model. The idea is to split the processing into actors, each of which can be processing a set of data or tasked with a specific responsibility. The actors can be running on different machines, or even in different data centers, and they can be assigned to different tasks. The framework will automatically keep track of the data sets and will make sure that the data is processed in the correct order. The advantage of this approach is that it can handle a lot of data very efficiently. It also allows MSC to maintain a high degree of control over the processing, as they can monitor and manage the actors. Additionally, the digital factory can be easily scaled up or down as needed.
This new framework is being built around the old system to gradually take its place, as described in the strangler fig metaphor: as with this Australian tree, where its seeds start from the branches and grow to the ground, “strangling” the host, so the new framework is slowly taking shape to replace the old monolith pipeline. While the change is not sudden, after a while the previous infrastructure will be completely replaced by the new one.
What is the Actor model in detail?
The Actor Model is a programming model used to build concurrent, scalable, fault-tolerant systems. It is an alternative to the traditional threaded model of concurrency, which is difficult to use and often leads to errors. The Actor Model is based on the concept of an actor, which is an independent unit of execution with its own mailbox (message queue). Actors communicate with each other by sending messages, which are placed in the mailbox of the recipient actor. An actor can process only one message at a time, but it can receive messages from multiple other actors. The Actor Model has been used to build successful systems such as Erlang, Akka, and Orleans. In the Digital Factory, the Actor Model is used to build a distributed, scalable, fault-tolerant system that can process large amounts of data in real time.
The Actor Model has many benefits over the traditional threaded model of concurrency:
– It is easy to use and understand.
– It is more scalable. It is more fault-tolerant.
– It is more efficient.
Using Akka.NET
MSC decided to rely on Akka.NET to build its actor model. Akka.NET is a toolkit and runtime for building highly concurrent, distributed, and fault-tolerant event-driven applications on the .NET platform. Akka.NET is inspired by the Akka toolkit for the JVM, which has been used in production by companies like Twitter, LinkedIn, and Comcast. Akka.NET enables developers to build systems that are more responsive, resilient, and elastic. These systems are easier to develop, test, and deploy because they are loosely coupled and message-driven. In Akka.NET, actors are implemented as lightweight processes that are isolated from one another. Akka.NET provides a simple, yet powerful, actor model that makes it easy to create and manage actors. It also provides a rich set of tools for monitoring and managing actor-based systems.
Akka.NET allows you to focus on meeting business needs instead of writing low-level code to provide reliable behavior, fault tolerance, and high performance. In particular, Akka.NET has the following advantages:
- Multi-threaded behavior without the use of low-level concurrency constructs like atomics or locks.
In the words of Rob Pike “Do not communicate by sharing memory; instead, share memory by communicating.” You do not even need to think about memory visibility issues. One of the benefits of using the actor model is that it allows for multi-threaded behavior without the need for low-level concurrency constructs like locks and atomics. This can simplify your code and make it easier to reason about. Additionally, you don’t need to worry about memory visibility issues, as messages sent between actors are guaranteed to be processed in the order in which they were sent.
- Transparent remote communication between systems and their components. Another benefit of using the actor model is that it makes transparent remote communication between systems and components easy. You don’t need to write or maintain difficult networking code. All you need to do is send a message to an actor’s mailbox, and it will be delivered regardless of whether the actor is on the same machine or not.
- A clustered, high-availability architecture that is elastic, scales in or out, on demand
The actor model built with Akka.NET is also well suited for building clustered, high-availability applications. This is because an actor system can be easily scaled up or down as needed, and because actors can be migrated between nodes in a cluster with no downtime. Additionally, it makes it easy to add fault-tolerance to your applications.
- Testing framework as dedicated component part of akka.net ecosystem Akka.NET comes with a dedicated testing framework that makes it easy to test your actor-based applications. This testing framework is a part of the akka.NET ecosystem, and it is designed to work seamlessly with the actor model.
- Dedicated component for implementing Stream Processing apps
Akka.NET also has a dedicated component for implementing stream processing applications. This component is called Akka Streams, and it makes it easy to build concurrent and distributed stream processing applications.
- Connectors for the majority MoM backing services
Akka.NET comes with connectors for the majority of popular message-oriented middleware (MoM) systems. This makes it easy to integrate akka.NET applications with existing MoM systems.
Conclusion
Cloud architectures have quickly become a fundamental building block of the tech industry. Being able to design and develop complex cloud architectures is imperative for each company that wants to scale in this industry. Furthermore, new technologies and frameworks, like the described microservices-centric actor model are taking the place of older monolithic approaches. Companies that do not adapt to these new techniques can quickly become obsolete, or unable to deal with the ever-increasing amount of data they need to process. MSC has revolutionized its data pipeline to scale organically with the data it needed to process from its services. Learn more about the MSC framework and strategy, and how your company can adapt as well to cutting-edge cloud technologies.
Discover more about this topic in the Codemotion Meetup with MSC