Author Archives: Shanoj

Unknown's avatar

About Shanoj

Author : Shanoj is a Data engineer and solutions architect passionate about delivering business value and actionable insights through well-architected data products. He holds several certifications on AWS, Oracle, Apache, Google Cloud, Docker, Linux and focuses on data engineering and analysis using SQL, Python, BigData, RDBMS, Apache Spark, among other technologies. He has 17+ years of history working with various technologies in the Retail and BFS domains.

Microservices Architectures: The SAGA Pattern

Leave a reply

The Saga pattern is an architectural pattern utilized for managing distributed transactions in microservices architectures. It ensures data consistency across multiple services without relying on distributed transactions, which can be complex and inefficient in a microservices environment.

Key Concepts of the Saga Pattern

In the Saga pattern, a business process is broken down into a series of local transactions. Each local transaction updates the database and publishes an event or message to trigger the next transaction in the sequence. This approach helps maintain data consistency across services by ensuring that each step is completed before moving to the next one.

Types of Saga Patterns

There are several variations of the Saga pattern, each suited to different scenarios:

Choreography-based Saga: Each service listens for events and decides whether to proceed with the next step based on the events it receives. This decentralized approach is useful for loosely coupled services.

Orchestration-based Saga: A central coordinator, known as the orchestrator, manages the sequence of actions. This approach provides a higher level of control and is beneficial when precise coordination is required.

State-based Saga: Uses a shared state or state machine to track the progress of a transaction. Microservices update this state as they execute their actions, guiding subsequent steps.

Reverse Choreography Saga: An extension of the Choreography-based Saga where services explicitly communicate about how to compensate for failed actions.

Event-based Saga: Microservices react to events generated by changes in the system, performing necessary actions or compensations asynchronously.

Challenges Addressed by the Saga Pattern

The Saga pattern solves the problem of maintaining data consistency across multiple microservices in distributed transactions. It addresses several key challenges that arise in microservices architectures:

Distributed Transactions: In a microservices environment, a single business transaction often spans multiple services, each with its own database. Traditional ACID transactions don’t work well in this distributed context.

Data Consistency: Ensuring data consistency across different services and their databases is challenging when you can’t use a single, atomic transaction.

Scalability and Performance: Two-phase commit (2PC) protocols, which are often used for distributed transactions, can lead to performance issues and reduced scalability in microservices architectures.

Solutions Provided by the Saga Pattern

The Saga pattern solves these problems by:

  • Breaking down distributed transactions into a sequence of local transactions, each handled by a single service.
  • Using compensating transactions to undo changes if a step in the sequence fails, ensuring eventual consistency.
  • Flexibility in transaction management, allowing services to be added, modified, or removed without significantly impacting the overall transactional flow.
  • Better scalability by allowing each service to manage its own local transaction independently.
  • Improving fault tolerance by providing mechanisms to handle and recover from failures in the transaction sequence.
  • Visibility into the transaction process, which aids in debugging, auditing, and compliance.

Implementation Approaches

Choreography-Based Sagas

  • Decentralized Control: Each service involved in the saga listens for events and reacts to them independently, without a central controller.
  • Event-Driven Communication: Services communicate by publishing and subscribing to events.
  • Autonomy and Flexibility: Services can be added, removed, or modified without significantly impacting the overall system.
  • Scalability: Choreography can handle complex and frequent interactions more flexibly, making it suitable for highly scalable systems.

Orchestration-Based Sagas

  • Centralized Control: A central orchestrator manages the sequence of transactions, directing each service on what to do and when.
  • Command-Driven Communication: The orchestrator sends commands to services to perform specific actions.
  • Visibility and Control: The orchestrator has a global view of the saga, making it easier to manage and troubleshoot.

Choosing Between Choreography and Orchestration

When to Use Choreography

  • When you want to avoid creating a single point of failure.
  • When services need to be highly autonomous and independent.
  • When adding or removing services without disrupting the overall flow is a priority.

When to Use Orchestration

  • When you need to guarantee a specific order of execution.
  • When centralized control and visibility are crucial for managing complex workflows.
  • When you need to manage the lifecycle of microservices execution centrally.

Hybrid Approach

In some cases, a combination of both approaches can be beneficial. Choreography can be used for parts of the saga that require high flexibility and autonomy, while orchestration can manage parts that need strict control and coordination.

Challenges and Considerations

  • Complexity: Implementing SAGA can be more complex than traditional transactions.
  • Lack of Isolation: Intermediate states are visible, which can lead to consistency issues.
  • Error Handling: Designing and implementing compensating transactions can be tricky.
  • Testing: Thorough testing of all possible scenarios is crucial but can be challenging.

The Saga pattern is powerful for managing distributed transactions in microservices architectures, offering a balance between consistency, scalability, and resilience. By carefully selecting the appropriate implementation approach, organizations can effectively address the challenges of distributed transactions and maintain data consistency across their services.

Stackademic 🎓

Thank you for reading until the end. Before you go:

Apache Hive 101: Enabling ACID Transactions

Leave a reply

To create ACID tables, ensure Hive is configured to support ACID transactions by setting the following properties:

SET hive.support.concurrency=true;
SET hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
SET hive.compactor.initiator.on=true;
SET hive.compactor.worker.threads=1;

Hive Version Compatibility

ACID transactions in Hive are supported from version 0.14.0 onwards. Ensure that your Hive installation is at least this version. Hive 3.x introduced significant improvements and additional features for ACID transactions.

Creating ACID Tables

Full ACID Table

Full ACID tables support all CRUD (Create, Retrieve, Update, Delete) operations and require the ORC file format:

CREATE TABLE acidtbl (
    key int,
    value string
) 
STORED AS ORC 
TBLPROPERTIES ("transactional"="true");

Insert-Only ACID Table

Insert-only ACID tables support only insert operations and can use various storage formats:

CREATE TABLE acidtbl_insert_only (
    key int,
    value string
) 
STORED AS TEXTFILE 
TBLPROPERTIES ("transactional"="true", "transactional_properties"="insert_only");

Converting Tables to ACID

Non-ACID to Full ACID

To convert a non-ACID managed table to a full ACID table (requires ORC format):

ALTER TABLE nonacidtbl SET TBLPROPERTIES ('transactional'='true');

Non-ACID to Insert-Only ACID

To convert a non-ACID managed table to an insert-only ACID table:

ALTER TABLE nonacidtbl SET TBLPROPERTIES ('transactional'='true', 'transactional_properties'='insert_only');

Data Operations on ACID Tables

Inserting Data

INSERT INTO acidtbl VALUES (1, 'a');
INSERT INTO acidtbl VALUES (2, 'b');

Updating Data

UPDATE acidtbl SET value='updated' WHERE key=1;

Performing Merge Operations

MERGE INTO acidtbl USING src
ON acidtbl.key = src.key
WHEN MATCHED AND src.value IS NULL THEN DELETE
WHEN MATCHED AND (acidtbl.value != src.value) THEN UPDATE SET value = src.value
WHEN NOT MATCHED THEN INSERT VALUES (src.key, src.value);

Understanding Table Structures

ACID Tables (Transactional)

ACID tables have a specific directory structure in HDFS:

/user/hive/warehouse/t/
├── base_0000022/
│   └── bucket_00000
├── delta_0000023_0000023_0000/
│   └── bucket_00000
└── delta_0000024_0000024_0000/
    └── bucket_00000
  • Base Directory: Contains the original data files.
  • Delta Directories: Store changes (inserts, updates, deletes).

Non-ACID Tables

Non-ACID tables have a simpler structure:

/user/hive/warehouse/table_name/
├── file1.orc
├── file2.orc
└── file3.orc

# For partitioned tables:
/user/hive/warehouse/table_name/
├── partition_column=value1/
│   ├── file1.orc
│   └── file2.orc
└── partition_column=value2/
    ├── file3.orc
    └── file4.orc

File Format Considerations

  • Full ACID Tables: Only the ORC (Optimized Row Columnar) file format is supported for full ACID tables that allow all CRUD operations.
  • Insert-Only ACID Tables: These tables support various file formats, not limited to ORC. You can use formats like TEXTFILE, CSV, AVRO, or JSON.
  • Managed Tables: The managed table storage type is required for ACID tables.
  • External Tables: ACID properties cannot be applied to external tables, as changes to external tables are beyond Hive’s control.
  • Converting Existing Tables: When converting a non-ACID managed table to a full ACID table, the data must be in ORC format.
  • Default Format: When creating a full ACID table without specifying the storage format, Hive defaults to using ORC.

Managed vs. External Tables

  • Managed Tables: Support ACID transactions. They can be created as transactional (either full ACID or insert-only).

Example of a full ACID managed table:

CREATE TABLE managed_acid_table (
    id INT,
    name STRING
) 
STORED AS ORC 
TBLPROPERTIES ("transactional"="true");

Example of an insert-only ACID managed table:

CREATE TABLE managed_insert_only_table (
    id INT,
    name STRING
) 
STORED AS TEXTFILE 
TBLPROPERTIES ("transactional"="true", "transactional_properties"="insert_only");

External Tables: Do not support ACID transactions. These tables are used for data managed outside Hive’s control.

Example of an external table:

CREATE EXTERNAL TABLE external_table (
    id INT,
    name STRING
) 
STORED AS TEXTFILE 
LOCATION '/path/to/external/data';

Limitations of ACID Tables

  1. Performance Overhead: ACID tables introduce additional overhead due to the need for transactional logging and compaction processes.
  2. Storage Requirements: The delta files and base files can increase storage requirements.
  3. Compaction: Regular compaction is necessary to maintain performance and manage storage, which can add complexity.
  4. Version Dependency: Ensure that you are using a Hive version that supports the desired ACID features, as improvements and bug fixes are version-dependent.
  5. External Table Limitation: ACID properties cannot be applied to external tables.

Key Points

  1. Only managed tables can be converted to ACID tables.
  2. External tables cannot be made transactional.
  3. Full ACID tables require the ORC file format.
  4. Converting ACID tables back to non-ACID tables is not supported.

Stackademic 🎓

Thank you for reading until the end. Before you go:

Bulkhead Architecture Pattern: Data Security & Governance

Leave a reply

Today during an Azure learning session focused on data security and governance, our instructor had to leave unexpectedly due to a personal emergency. Reflecting on the discussion and drawing from my background in fintech and solution architecture, I believe it would be beneficial to explore an architecture pattern relevant to our conversation: the Bulkhead Architecture Pattern.

Inspired by ship design, the Bulkhead architecture pattern divides the base of a ship into partitions called bulkheads. This ensures that if there’s a leak in one section, it doesn’t sink the entire ship; only the affected partition fills with water. Translating this principle to software architecture, the pattern focuses on fault isolation by decomposing a monolithic architecture into a microservices architecture.

Use Case: Bank Reconciliation Reporting

Consider a scenario involving trade data across various regions such as APAC, EMEA, LATAM, and NAM. Given the regulatory challenges related to cross-country data movement, ensuring proper data governance when consolidating data in a data warehouse environment becomes crucial. Specifically, it is essential to manage the challenge of ensuring that data from India cannot be accessed from the NAM region and vice versa. Additionally, restricting data movement at the data centre level is critical.

Microservices Isolation

  • Microservices A, B, C: Each microservice is deployed in its own Azure Kubernetes Service (AKS) cluster or Azure App Service.
  • Independent Databases: Each microservice uses a separate database instance, such as Azure SQL Database or Cosmos DB, to avoid single points of failure.

Network Isolation

  • Virtual Networks (VNets): Each microservice is deployed in its own VNet. Use Network Security Groups (NSGs) to control inbound and outbound traffic.
  • Private Endpoints: Secure access to Azure services (e.g., storage accounts, databases) using private endpoints.

Load Balancing and Traffic Management

  • Azure Front Door: Provides global load balancing and application acceleration for microservices.
  • Application Gateway: Offers application-level routing and web application firewall (WAF) capabilities.
  • Traffic Manager: A DNS-based traffic load balancer for distributing traffic across multiple regions.

Service Communication

  • Service Bus: Use Azure Service Bus for decoupled communication between microservices.
  • Event Grid: Event-driven architecture for handling events across microservices.

Fault Isolation and Circuit Breakers

  • Polly: Implement circuit breakers and retries within microservices to handle transient faults.
  • Azure Functions: Use serverless functions for non-critical, independently scalable tasks.

Data Partitioning and Isolation

  • Sharding: Partition data across multiple databases to improve performance and fault tolerance.
  • Data Sync: Use Azure Data Sync to replicate data across regions for redundancy.

Monitoring and Logging

  • Azure Monitor: Centralized monitoring for performance and availability metrics.
  • Application Insights: Deep application performance monitoring and diagnostics.
  • Log Analytics: Aggregated logging and querying for troubleshooting and analysis.

Advanced Threat Protection

  • Azure Defender for Storage: Enable Azure Defender for Storage to detect unusual and potentially harmful attempts to access or exploit storage accounts.

Key Points

  • Isolation: Each microservice and its database are isolated in separate clusters and databases.
  • Network Security: VNets and private endpoints ensure secure communication.
  • Resilience: Circuit breakers and retries handle transient faults.
  • Monitoring: Centralized monitoring and logging for visibility and diagnostics.
  • Scalability: Each component can be independently scaled based on load.

Bulkhead Pattern Concepts

Isolation

The primary goal of the Bulkhead pattern is to isolate different parts of a system to contain failures within a specific component, preventing them from cascading and affecting the entire system. This isolation can be achieved through various means such as separate thread pools, processes, or containers.

Fault Tolerance

By containing faults within isolated compartments, the Bulkhead pattern enhances the system’s ability to tolerate failures. If one component fails, the rest of the system can continue to operate normally, thereby improving overall reliability and stability.

Resource Management

The pattern helps in managing resources efficiently by allocating specific resources (like CPU, memory, and network bandwidth) to different components. This prevents resource contention and ensures that a failure in one component does not exhaust resources needed by other components.

Implementation Examples in K8s

Kubernetes

An example of implementing the Bulkhead pattern in Kubernetes involves creating isolated containers for different services, each with its own CPU and memory resources and limits. This configuration is for a service called payment-processing.

apiVersion: v1
kind: Pod
metadata:
  name: payment-processing
spec:
  containers:
  - name: payment-processing-container
    image: payment-service:latest
    resources:
      requests:
        memory: "128Mi"
        cpu: "500m"
      limits:
        memory: "256Mi"
        cpu: "2"
---
apiVersion: v1
kind: Pod
metadata:
  name: order-management
spec:
  containers:
  - name: order-management-container
    image: order-service:latest
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "1"
---
apiVersion: v1
kind: Pod
metadata:
  name: inventory-control
spec:
  containers:
  - name: inventory-control-container
    image: inventory-service:latest
    resources:
      requests:
        memory: "96Mi"
        cpu: "300m"
      limits:
        memory: "192Mi"
        cpu: "1.5"

In this configuration:

  • The payment-processing service is allocated 128Mi of memory and 500m of CPU as a request, with limits set to 256Mi of memory and 2 CPUs.
  • The order-management service has its own isolated resources, with 64Mi of memory and 250m of CPU as a request, and limits set to 128Mi of memory and 1 CPU.
  • The inventory-control service is given 96Mi of memory and 300m of CPU as a request, with limits set to 192Mi of memory and 1.5 CPUs.

This setup ensures that each service operates within its own resource limits, preventing any single service from exhausting resources and affecting the others.

Hystrix

Hystrix, a Netflix API for latency and fault tolerance, uses the Bulkhead pattern to limit the number of concurrent calls to a component. This is achieved through thread isolation, where each component is assigned a separate thread pool, and semaphore isolation, where callers must acquire a permit before making a request. This prevents the entire system from becoming unresponsive if one component fails.

Ref: https://github.com/Netflix/Hystrix

AWS App Mesh

In AWS App Mesh, the Bulkhead pattern can be implemented at the service-mesh level. For example, in an e-commerce application with different API endpoints for reading and writing prices, resource-intensive write operations can be isolated from read operations by using separate resource pools. This prevents resource contention and ensures that read operations remain unaffected even if write operations experience a high load.

Benefits

  • Fault Containment: Isolates faults within specific components, preventing them from spreading and causing systemic failures.
  • Improved Resilience: Enhances the system’s ability to withstand unexpected failures and maintain stability.
  • Performance Optimization: Allocates resources more efficiently, avoiding bottlenecks and ensuring consistent performance.
  • Scalability: Allows independent scaling of different components based on workload demands.
  • Security Enhancement: Reduces the attack surface by isolating sensitive components, limiting the impact of security breaches.

The Bulkhead pattern is a critical design principle for constructing resilient, fault-tolerant, and efficient systems by isolating components and managing resources effectively.

Stackademic 🎓

Thank you for reading until the end. Before you go:

Apache Hive 101: MSCK Repair Table

Leave a reply

The MSCK REPAIR TABLE command in Hive is used to update the metadata in the Hive metastore to reflect the current state of the partitions in the file system. This is particularly necessary for external tables where partitions might be added directly to the file system (such as HDFS or Amazon S3) without using Hive commands.

What MSCK REPAIR TABLE Does

  1. Scans the File System: It scans the file system (e.g., HDFS or S3) for Hive-compatible partitions that were added after the table was created.
  2. Updates Metadata: It compares the partitions in the table metadata with those in the file system. If it finds new partitions in the file system that are not in the metadata, it adds them to the Hive metastore.
  3. Partition Detection: It detects partitions by reading the directory structure and creating partitions based on the folder names.

Why MSCK REPAIR TABLE is Needed

  1. Partition Awareness: Hive stores a list of partitions for each table in its metastore. When new partitions are added directly to the file system, Hive is not aware of these partitions unless the metadata is updated. Running MSCK REPAIR TABLE ensures that the Hive metastore is synchronized with the actual data layout in the file system.
  2. Querying New Data: Without updating the metadata, queries on the table will not include the data in the new partitions. By running MSCK REPAIR TABLE, you make the new data available for querying.
  3. Automated Ingestion: For workflows that involve automated data ingestion, running MSCK REPAIR TABLE after each data load ensures that the newly ingested data is recognized by Hive without manually adding each partition.

Command to Run MSCK REPAIR TABLE

MSCK REPAIR TABLE table_name;

Replace table_name with the name of your Hive table.

Considerations and Limitations

  1. Performance: The operation can be slow, especially with a large number of partitions, as it involves scanning the entire directory structure.
  2. Incomplete Updates: If the operation times out, it may leave the table in an incomplete state where only some partitions are added. It may be necessary to run the command multiple times until all partitions are included.
  3. Compatibility: MSCK REPAIR TABLE only adds partitions to metadata; it does not remove them. For removing partitions, other commands like ALTER TABLE DROP PARTITION must be used.
  4. Hive Compatibility: Partitions must be Hive-compatible. For partitions that are not, manual addition using ALTER TABLE ADD PARTITION is required.

Software Architecture: Space-Based Architecture Pattern

Leave a reply

Scaling an application is a challenging task. To scale effectively, you often need to increase the number of web servers, application servers, and database servers. However, this can make your architecture complex due to the need for high performance and scalability to serve thousands of concurrent users.

Horizontal scaling of the database layer typically involves sharding, which adds further complexity and makes it difficult to manage.

In general, Space-Based Architecture (SBA) addresses the challenge of creating highly scalable and elastic systems capable of handling a vast number of concurrent users and operations. Traditional architectures often struggle with performance bottlenecks due to direct interactions with the database for transactional data, leading to limitations in scalability and elasticity.

What is Space-Based Architecture (SBA)?

In Space-Based Architecture (SBA), you scale your application by removing the database and instead using memory grids to manage the data. Instead of scaling a particular tier in your application, you scale the entire architecture together as a unified process. SBA is widely used in distributed computing to increase the scalability and performance of a solution. This architecture is based on the concept of a tuple space.

Note: A tuple space is a shared memory object that provides operations to store and retrieve ordered sets of data, called tuples. It is an implementation of the associative memory paradigm for parallel/distributed computing, allowing multiple processes to access and manipulate tuples concurrently.

Goals of SBA

High Scalability and Elasticity:

· Efficiently managing and processing millions of concurrent users and transactions without direct database interactions.

· Enabling rapid scaling from a small number of users to hundreds of thousands or more within milliseconds.

Performance Optimization:

· Reducing latency by utilizing in-memory data grids and caching mechanisms instead of direct database reads and writes.

· Ensuring quick data access times measured in nanoseconds for a seamless user experience.

Eventual Consistency:

· Maintaining eventual consistency across distributed processing units through replicated caching and asynchronous data writes to the database.

Decoupling Database Dependency:

· Minimizing the dependency on the database for real-time transaction processing to prevent database bottlenecks and improve system responsiveness.

Handling High Throughput:

Managing high throughput demands without overwhelming the database by leveraging in-memory data replication and distributed processing units.

Key Components of SBA

Processing Units (PU):

These are individual nodes or containers that encapsulate the processing logic and the data they operate on. Each PU is responsible for executing business logic and can be replicated or partitioned for scalability and fault tolerance. They typically include web-based components, backend business logic, an in-memory data grid, and a replication engine.

Virtualized Middleware:

This layer handles shared infrastructure concerns and includes:

· Data Grid: A crucial component that allows requests to be assigned to any available processing unit, ensuring high performance and reliability. The data grid is responsible for synchronizing data between the processing units by building the tuple space.

· Messaging Grid: Manages the flow of incoming transactions and communication between services.

· Processing Grid: Enables parallel processing of events among different services based on the master/worker pattern.

· Deployment Manager: Manages the startup and shutdown of PUs, starts new PUs to handle additional load, and shuts down PUs when no longer needed.

· Data Pumps and Data Readers/Writers: Data pumps marshal data between the database and the processing units, ensuring consistent data updates across nodes.

Now you might be naturally thinking, what makes SBA different from a traditional memory cache database?

Differences Between SBA and Memory Cache Databases

Data Consistency:

· SBA: Uses an eventual consistency model, where updates are asynchronously propagated across nodes, ensuring eventual convergence without the need for immediate consistency, which can introduce significant performance overhead.

· Memory Cache Database: Typically uses strong consistency models, ensuring immediate consistency across all nodes, which can impact performance.

Scalability:

· SBA: Achieves linear scalability by adding more processing units (PUs) as needed, ensuring the system can handle increasing workloads without performance degradation.

· Memory Cache Database: Scalability is often limited by the underlying database architecture and can be more complex to scale horizontally.

Data Replication:

· SBA: Replicates data across multiple nodes to ensure fault tolerance and high availability. In the event of a node failure, the system can seamlessly recover by accessing replicated data from other nodes.

· Memory Cache Database: Data replication is used for performance and availability but can be more complex to manage and maintain consistency.

Data Grid:

· SBA: Utilizes a distributed data grid that allows requests to be assigned to any available processing unit, ensuring high performance and reliability.

· Memory Cache Database: Typically uses a centralized cache that can become a bottleneck as the system scales.

Processing:

· SBA: Enables parallel processing across multiple nodes, leading to improved throughput and response times.

· Memory Cache Database: Processing is typically done within the database or cache layer, which can be less scalable and efficient.

Deployment:

· SBA: Supports elastic scalability by adding or removing nodes as needed, ensuring the system can handle increased workloads without compromising performance or data consistency.

· Memory Cache Database: Deployment and scaling can be more complex and often require significant infrastructure changes.

Cost:

· SBA: Can be more cost-effective by leveraging distributed computing and in-memory processing, reducing the need for expensive hardware and infrastructure upgrades.

· Memory Cache Database: Can be more expensive due to the need for high-performance hardware and infrastructure to support the cache layer.

Now you might be wondering, is SBA suitable for every scenario?

Limitations of SBA

High Data Synchronization and Consistency Requirements:

Systems that require immediate data consistency and high synchronization across all components will not benefit from SBA due to its eventual consistency model.

The delay in synchronizing data with the database may not meet the needs of applications requiring real-time consistency.

Large Volumes of Transactional Data:

Applications needing to store and manage massive amounts of transactional data (e.g., terabytes) are not suitable for SBA.

Keeping such large volumes of data in memory is impractical and may exceed the memory capacity of available hardware.

Budget and Time Constraints:

Projects with strict budget and time constraints are likely to overrun their resources due to the technical complexity of implementing SBA.

The initial setup and implementation are resource-intensive, requiring significant investment in both time and money.

Technical Complexity:

The high technical complexity of SBA makes it challenging to implement, maintain, and troubleshoot.

Organizations lacking the necessary expertise and experience may find it difficult to manage the intricacies of SBA.

Cost Considerations:

The cost of maintaining in-memory data grids and replicated caching can be prohibitive, especially for smaller organizations or projects with limited budgets.

The infrastructure required to support SBA’s scalability and performance may be expensive to acquire and maintain.

Limited Agility:

SBA offers limited agility compared to other architectural styles due to its complex setup and eventual consistency model.

Changes and updates to the system may require significant effort and coordination across distributed processing units.

Now, let’s dive into some use cases and solutions that demonstrate the power of SBA.

Use Cases and Solutions

Space-Based Architecture (SBA) addresses several critical challenges that traditional architectures face, particularly in high-transaction, high-availability, and variable load environments.

Scalability Bottlenecks:

· Problem: Traditional architectures often struggle to scale horizontally due to limitations in centralized data storage and processing.

· Solution: SBA enables horizontal scalability by distributing processing units (PUs) across multiple nodes. Each PU can handle a portion of the workload independently, allowing the system to scale out by simply adding more PUs.

High Availability and Fault Tolerance:

· Problem: Ensuring high availability and fault tolerance is challenging in monolithic or tightly coupled systems.

· Solution: SBA enhances fault tolerance through redundancy and data replication. Each PU operates independently, and data is replicated across multiple PUs. If one PU fails, others can take over, ensuring continuous availability and minimal downtime.

Performance Issues:

· Problem: Traditional systems often rely heavily on relational databases, leading to performance bottlenecks due to slow disk I/O and limited scalability of single-node databases.

· Solution: SBA leverages in-memory data grids, which provide faster data access and reduce the dependency on disk-based storage, significantly improving response times and overall system performance.

Handling Variable and Unpredictable Loads:

· Problem: Many applications experience variable and unpredictable workloads, such as seasonal spikes in e-commerce or fluctuating traffic in social media platforms.

· Solution: SBA’s elastic nature allows it to automatically adjust to varying loads by adding or removing PUs as needed, ensuring the system can handle peak loads without performance degradation.

Reducing Single Points of Failure:

· Problem: Centralized components, such as single database servers or monolithic application servers, can become single points of failure.

· Solution: SBA decentralizes processing and storage, eliminating single points of failure. Each PU can function independently, and the system can continue to operate even if some PUs fail.

Complex Data Management:

· Problem: Managing large volumes of data and ensuring its consistency, availability, and partitioning across a distributed system can be complex.

· Solution: SBA uses distributed data stores and in-memory data grids to manage data efficiently, ensuring data consistency and availability through replication and partitioning strategies.

Simplifying Deployment and Maintenance:

· Problem: Deploying and maintaining traditional monolithic applications can be cumbersome.

· Solution: SBA’s modular nature simplifies deployment and maintenance. Each PU can be developed, tested, and deployed independently, reducing the risk of system-wide issues during updates or maintenance.

Latency and Real-Time Processing:

· Problem: Real-time processing and low-latency requirements are difficult to achieve with traditional architectures.

· Solution: SBA’s use of in-memory data grids and asynchronous messaging grids ensures low latency and real-time processing capabilities, crucial for applications requiring immediate data processing and response.

Space-Based Architecture addresses several significant challenges faced by traditional architectures, making it an ideal choice for applications requiring high scalability, performance, availability, and resilience. By distributing processing and data management across independent units, SBA ensures that systems can handle modern demands efficiently and effectively.

Stackademic 🎓

Thank you for reading until the end. Before you go:

Event-Driven Architecture (EDA)

Leave a reply

Event-Driven Architecture (EDA) is a software design paradigm that emphasizes producing, detecting, and reacting to events. Two important architectural concepts within EDA are:

Asynchrony

Asynchrony in EDA refers to the ability of services to communicate without waiting for immediate responses. This is crucial for building scalable and resilient systems. Here are key points about asynchrony:

  • Decoupled Communication: Services can send messages or events without needing to wait for a response, allowing them to continue processing other tasks. This decoupling enhances system performance and scalability.
  • Example: Service A invokes Service B with a request and receives a response asynchronously. Similarly, Service C submits a batch job to Service D and receives an acknowledgement, then polls for the job status and gets updates later

Event-Driven Communication

Event-driven communication is the core of EDA, where events trigger actions across different services. This approach ensures that systems can react to changes in real-time and remain loosely coupled. Key aspects include:

  • Event Producers and Consumers: Events are generated by producers and consumed by interested services. This model supports real-time processing and decoupling of services.
  • Example: Service C submits a batch job to Service D and receives an acknowledgement. Upon completion, Service D sends a notification to Service C, allowing it to react to the event without polling

Key Definitions

  • Event-driven architecture (EDA): Uses events to communicate between decoupled applications asynchronously.
  • Event Producer or Publisher: Generates events, such as account creation or deletion.
  • Event Broker: Receives events from producers and routes them to appropriate consumers.
  • Event Consumer or Subscriber: Receives and processes events from the broker.

Characteristics of Event Components

Event Producer:

  • Agnostic of consumers
  • Adds producer’s identity
  • Conforms to a schema
  • Unique event identifier
  • Adds just the required data

Event Consumer:

  • Idempotent (can handle duplicate events without adverse effects)
  • Ordering not guaranteed
  • Ensures event authenticity
  • Stores events and processes them

Event Broker:

  • Handles multiple publishers and subscribers
  • Routes events to multiple targets
  • Supports event transformation
  • Maintains a schema repository

Important Concepts

  • Event: Something that has already happened in the system.
  • Service Choreography: A coordinated sequence of actions across multiple microservices to accomplish a business process. It promotes service decoupling and asynchrony, enabling extensibility.

Common Mistakes

Overly complex event-driven designs can lead to tangled architectures.

Overly complex event-driven designs can lead to tangled architectures, which are difficult to manage and maintain. Here are some real-world examples and scenarios illustrating this issue:

Example 1: Microservices Overload

In a large-scale microservices architecture, each service may generate and process numerous events. For example, an e-commerce platform might include services for inventory, orders, payments, shipping, and notifications. If each of these services creates events for every change in state and processes events from various other services, the number of event interactions can grow significantly. This can result in a scenario where:

  • Event Storming: Too many events are being produced and consumed, making it hard to track which service is responsible for what.
  • Service Coupling: Services become tightly coupled through their event dependencies, making it difficult to change one service without impacting others.
  • Debugging Challenges: Tracing the flow of events to diagnose issues becomes complex, as events might trigger multiple services in unpredictable ways.

Example 2: Financial Transactions

In a financial system, different services might handle account management, transaction processing, fraud detection, and customer notifications. If these services are designed to emit and listen to numerous events, the architecture can become tangled:

  • Complex Event Chains: A single transaction might trigger a cascade of events across multiple services, making it hard to ensure data consistency and integrity.
  • Latency Issues: The time taken for events to propagate through the system can introduce latency, affecting the overall performance.
  • Security Concerns: With multiple services accessing and emitting sensitive financial data, ensuring secure communication and data integrity becomes more challenging.

Example 3: Healthcare Systems

In a healthcare system, services might handle patient records, appointment scheduling, billing, and notifications. An overly complex event-driven design can lead to:

  • Data Inconsistency: If events are not processed in the correct order or if there are failures in event delivery, patient data might become inconsistent.
  • Maintenance Overhead: Keeping track of all the events and ensuring that each service is correctly processing them can become a significant maintenance burden.
  • Regulatory Compliance: Ensuring that the system complies with healthcare regulations (e.g., HIPAA) can be more difficult when data is flowing through numerous services and events.

Mitigation Strategies

To avoid these pitfalls, it is essential to:

  • Simplify Event Flows: Design events at the right level of abstraction and avoid creating too many fine-grained events.
  • Clear Service Boundaries: Define clear boundaries for each service and ensure that events are only produced and consumed within those boundaries.
  • Use Event Brokers: Employ event brokers or messaging platforms to decouple services and manage event routing more effectively.
  • Invest in Observability: Implement robust logging, monitoring, and tracing to track the flow of events and diagnose issues quickly.

“Simplicity is the soul of efficiency.” — Austin Freeman

By leveraging asynchrony and event-driven communication, EDA enables the construction of robust, scalable, and flexible systems that can handle complex workflows and real-time data processing.

Stackademic 🎓

Thank you for reading until the end. Before you go:

Microservice 101: Micro Frontend Architecture Pattern

Leave a reply

The Micro Frontend Architecture Pattern is a design approach that entails breaking down a large web application into smaller, independent front-end applications. Each of these applications is responsible for a specific part of the user interface. This approach draws inspiration from microservices architecture and aims to deliver similar benefits, such as scalability, faster development times, and improved resource management.

Key Points

  • Decomposition: Break down a large web application into smaller, independent front-end applications.
  • Autonomy: Each front-end application is responsible for a specific part of the UI and can be developed, deployed, and maintained independently.
  • Scalability: Micro frontends can be scaled up or down independently, allowing for more efficient resource allocation.
  • Faster Development: Independent development teams can work on different micro frontends simultaneously, reducing development time.
  • Better Resource Management: Micro frontends can be optimized for specific tasks, reducing the load on the server and improving performance.

Types of Micro Frontend Patterns

  • Component Library Pattern: A centralized library of reusable components that can be used across multiple micro frontends.
  • Component Sharing Pattern: Micro frontends share components, reducing duplication and improving consistency.
  • Route-Based Pattern: Micro frontends are organized based on routes, with each route handling a specific part of the UI.
  • Event-Driven Pattern: Micro frontends communicate with each other through events, allowing for loose coupling and greater flexibility.
  • Iframe-Based Pattern: Micro frontends are embedded in separate iframes, providing isolation and reducing conflicts.
  • Server-Side Rendering Pattern: The server assembles the HTML and components of multiple micro frontends into a single page, reducing client-side complexity.

Advantages

  • Improved Scalability: Micro frontends can be scaled up or down independently, allowing for more efficient resource allocation.
  • Faster Development: Independent development teams can work on different micro frontends simultaneously, reducing development time.
  • Better Resource Management: Micro frontends can be optimized for specific tasks, reducing the load on the server and improving performance.
  • Enhanced Autonomy: Each micro frontend can be developed, deployed, and maintained independently, allowing for greater autonomy and flexibility.

Challenges

  • Complexity: Micro frontends can introduce additional complexity, especially when integrating multiple micro frontends.
  • Communication: Micro frontends need to communicate with each other, which can be challenging, especially in event-driven patterns.
  • Testing: Testing micro frontends can be more complex due to the distributed nature of the architecture.

Tools and Technologies

  • Bit: A platform that allows for building, sharing, and reusing components across micro frontends.
  • Client-Side Composition: A technique that uses client-side scripting to assemble the HTML and components of multiple micro frontends.
  • Server-Side Rendering: A technique that uses server-side rendering to assemble the HTML and components of multiple micro frontends into a single page.

Examples

  • Amazon: Uses micro frontends to manage different parts of its UI, such as search and recommendations.
  • Zalando: Uses micro frontends to manage different parts of its e-commerce platform, such as product listings and checkout.
  • Capital One: Uses micro frontends to manage different parts of its banking platform, such as account management and transactions.

The Micro Frontends Architecture Pattern is an effective approach for creating scalable, maintainable, and efficient web applications. It involves breaking down a large application into smaller, independent front-end applications. This approach helps developers work more efficiently, reduce complexity, and improve performance. However, it requires careful planning, communication, and testing to ensure seamless integration and achieve optimal results.

Stackademic 🎓

Thank you for reading until the end. Before you go:

Microservice 101: The Strangler Fig pattern

Leave a reply

The Strangler Fig pattern is a design pattern used in microservices architecture to gradually replace a monolithic application with microservices. It is named after the Strangler Fig tree, which grows around a host tree, eventually strangling it. In this pattern, new microservices are developed alongside the existing monolithic application, gradually replacing its functionality until the monolith is no longer needed.

Key Steps

  1. Transform: Identify a module or functionality within the monolith to be replaced by a new microservice. Develop the microservice in parallel with the monolith.
  2. Coexist: Implement a proxy or API gateway to route requests to either the monolith or the new microservice. This allows both systems to coexist and ensures uninterrupted functionality.
  3. Eliminate: Gradually shift traffic from the monolith to the microservice. Once the microservice is fully functional, the monolith can be retired.

Advantages

  • Incremental Migration: Minimizes risks associated with complete system rewrites.
  • Flexibility: Allows for independent development and deployment of microservices.
  • Reduced Disruptions: Ensures uninterrupted system functionality during the migration process.

Disadvantages

  • Complexity: Requires careful planning and coordination to manage both systems simultaneously.
  • Additional Overhead: Requires additional resources for maintaining both the monolith and the microservices.

Implementation

  1. Identify Module: Select a module or functionality within the monolith to be replaced.
  2. Develop Microservice: Create a new microservice to replace the identified module.
  3. Implement Proxy: Configure an API gateway or proxy to route requests to either the monolith or the microservice.
  4. Gradual Migration: Shift traffic from the monolith to the microservice incrementally.
  5. Retire Monolith: Once the microservice is fully functional, retire the monolith.

Tools and Technologies

  • API Gateway: Used to route requests to either the monolith or the microservice.
  • Change Data Capture (CDC): Used to stream changes from the monolith to the microservice.
  • Event Streaming Platform: Used to create event streams that can be used by other applications.

Examples

  • E-commerce Application: Migrate order management functionality from a monolithic application to microservices using the Strangler Fig pattern.
  • Legacy System: Use the Strangler Fig pattern to gradually replace a legacy system with microservices.

The Strangler Fig pattern is a valuable tool for migrating monolithic applications to microservices. It allows for incremental migration, reduces disruptions, and minimizes risks associated with complete system rewrites. However, it requires careful planning and coordination to manage both systems simultaneously.

Stackademic 🎓

Thank you for reading until the end. Before you go:

Solution Architect: Different Methodologies

Leave a reply

This article is an outcome of a discussion with a fellow solution architect. We were discussing the different approaches or schools of thought a solution architect might follow. If there is some disagreement, we kindly ask that you respect our point of view, and we are open to any kind of healthy discussion on this topic.

“Good architecture is like a great novel: it gets better with every reading.” — Robert C. Martin

In the field of solution architecture, there are several approaches one might take. Among them are the Problem-First Approach, Design-First Approach, Domain-Driven Design (DDD), and Agile Architecture. Each has its own focus and methodology, and the choice of approach depends on the context and specific needs of the project.

“The goal of software architecture is to minimize the human resources required to build and maintain the required system.” — Robert C. Martin

Based on the various approaches discussed, we propose a common and effective order for a solution architect to follow:

1. Problem Statement

Define and Understand the Problem: Begin by clearly defining the problem that needs to be solved. This involves gathering requirements, understanding business needs, objectives, constraints, and identifying any specific challenges. This foundational step ensures that all subsequent efforts are aligned with solving the correct issue.

“In software, the most beautiful code, the most beautiful functions, and the most beautiful programs are sometimes not there at all.” — Jon Bentley

2. High-Level Design

Develop a Conceptual Framework: Create a high-level design that outlines the overall structure of the solution. Identify major components, their interactions, data flow, and the overall system architecture. This step provides a bird’s-eye view of the solution, ensuring that all stakeholders have a common understanding of the proposed system.

“The most important single aspect of software development is to be clear about what you are trying to build.” — Bjarne Stroustrup

3. Architecture Patterns

Select Suitable Patterns: Identify and choose appropriate architecture patterns that fit the high-level design and problem context. Patterns such as microservices, layered architecture, and event-driven architecture help ensure the solution is robust, scalable, and maintainable. Selecting the right pattern is crucial for addressing the specific needs and constraints of the project.

“A pattern is a solution to a problem in a context.” — Christopher Alexander

4. Technology Stacks

Choose Technologies: Select the technology stacks that will be used to implement the solution. This includes programming languages, frameworks, databases, cloud services, and other tools that align with the architecture patterns and high-level design. Consider factors like team expertise, performance, scalability, and maintainability. The choice of technology stack has a significant impact on the implementation and long-term success of the project.

“Any sufficiently advanced technology is indistinguishable from magic.” — Arthur C. Clarke

5. Low-Level Design

Detail Each Component: Create detailed, low-level designs for each component identified in the high-level design. Specify internal structures, interfaces, data models, algorithms, and detailed workflows. This step ensures that each component is well-defined and can be effectively implemented by development teams. Detailed design documents help in minimizing ambiguities and ensuring a smooth development process.

“Good design adds value faster than it adds cost.” — Thomas C. Gale

Summary of Order:

Practical Considerations:

  • Iterative Feedback and Validation: Incorporate iterative feedback and validation throughout the process. Regularly review designs with stakeholders and development teams to ensure alignment with business goals and to address any emerging issues. This iterative process helps in refining the solution and addressing any unforeseen challenges.

“You can’t improve what you don’t measure.” — Peter Drucker

  • Documentation: Maintain comprehensive documentation at each stage to ensure clarity and facilitate communication among stakeholders. Good documentation practices help in maintaining a record of decisions and the rationale behind them, which is useful for future reference and troubleshooting.
  • Flexibility: Be prepared to adapt and refine designs as new insights and requirements emerge. This approach allows for continuous improvement and alignment with evolving business needs. Flexibility is key to responding effectively to changing business landscapes and technological advancements.

“The measure of intelligence is the ability to change.” — Albert Einstein

Guidelines for Selecting an Approach

Here are some general guidelines for selecting an approach:

Problem-First Approach: This approach is suitable when the problem domain is well-understood, and the focus is on finding the best solution to address the problem. It works well for projects with clear requirements and constraints.

Design-First Approach: This approach is beneficial when the system’s architecture and design are critical, and upfront planning is necessary to ensure the system meets its quality attributes and non-functional requirements.

Domain-Driven Design (DDD): DDD is a good fit for complex domains with intricate business logic and evolving requirements. It promotes a deep understanding of the domain and helps in creating a maintainable and extensible system.

Agile Architecture: An agile approach is suitable when requirements are likely to change frequently, and the team needs to adapt quickly. It works well for projects with a high degree of uncertainty or rapidly changing business needs.

Ultimately, the choice of approach should be based on a careful evaluation of the project’s specific context, requirements, and constraints, as well as the team’s expertise and the organization’s culture and processes. It’s also common to combine elements from different approaches or tailor them to the project’s needs.

“The best way to predict the future is to invent it.” — Alan Kay

Real-Life Use Case: Netflix Microservices Architecture

A notable real-life example of following a structured approach in solution architecture is Netflix’s transition to a microservices architecture. Here’s how Netflix applied a similar order in their architectural approach:

1. Problem Statement

Netflix faced significant challenges with their existing monolithic architecture, including scalability issues, difficulty in deploying new features, and handling increasing loads as their user base grew globally. The problem was clearly defined: the need for a scalable, resilient, and rapidly deployable architecture to support their expanding services.

“If you define the problem correctly, you almost have the solution.” — Steve Jobs

2. High-Level Design

Netflix designed a high-level architecture that focused on breaking down their monolithic application into smaller, independent services. This conceptual framework provided a clear vision of how different components would interact and be managed. They aimed to achieve a highly decoupled system where services could be developed and deployed independently.

3. Architecture Patterns

Netflix chose a combination of several architectural patterns to meet their specific needs:

  • Microservices Architecture: This pattern allowed Netflix to create independent services that could be developed, deployed, and scaled individually. Each microservice handled a specific business capability and communicated with others through well-defined APIs. This pattern provided the robustness and scalability needed to handle millions of global users.
  • Event-Driven Architecture: Netflix implemented an event-driven architecture to handle asynchronous communication between services. This pattern was essential for maintaining responsiveness and reliability in a highly distributed system. Services are communicated via events, allowing the system to remain loosely coupled and scalable.

Ref: https://github.com/Netflix/Hystrix

  • Circuit Breaker Pattern: Using tools like Hystrix, Netflix adopted the circuit breaker pattern to prevent cascading failures and to manage service failures gracefully. This pattern improved the resilience and fault tolerance of their architecture.
  • Service Discovery Pattern: Netflix utilized Eureka for service discovery. This pattern ensured that services could dynamically locate and communicate with each other, facilitating load balancing and failover strategies.
  • API Gateway Pattern: Zuul was employed as an API gateway, providing a single entry point for all client requests. This pattern helped manage and route requests to the appropriate microservices, improving security and performance.

4. Technology Stacks

Netflix selected a technology stack that included:

  • Java: For developing the core services due to its maturity, scalability, and extensive ecosystem.
  • Cassandra: For data storage, providing high availability and scalability across multiple data centers.
  • AWS: For cloud infrastructure, offering scalability, reliability, and a wide range of managed services.

Netflix also implemented additional tools and technologies to support their architecture patterns:

  • Hystrix: For implementing the circuit breaker pattern.
  • Eureka: For service discovery and registration.
  • Zuul: For API gateway and request routing.
  • Kafka: For event-driven messaging and real-time data processing.
  • Spinnaker: For continuous delivery and deployment automation.

5. Low-Level Design

Detailed designs for each microservice were created, specifying how they would interact with each other, handle data, and manage failures. This included defining:

  • APIs: Well-defined interfaces for communication between services.
  • Data Models: Schemas and structures for data storage and exchange.
  • Communication Protocols: RESTful APIs, gRPC, and event-based messaging.
  • Internal Structures: Detailed workflows, algorithms, and internal component interactions.

Each microservice was developed with clear boundaries and responsibilities, ensuring a well-structured implementation. Teams were organized around microservices, allowing for autonomous development and deployment cycles.

“The details are not the details. They make the design.” — Charles Eames

Practical Considerations

Netflix continuously incorporated iterative feedback and validation through extensive testing and monitoring. They maintained comprehensive documentation for their microservices, facilitating communication and understanding among teams. Flexibility was a core principle, allowing Netflix to adapt and refine their services based on real-time performance data and user feedback.

  • Iterative Feedback and Validation: Netflix used canary releases, A/B testing, and real-time monitoring to gather feedback and validate changes incrementally. This allowed them to make informed decisions and continuously improve their services.

Ref: https://netflixtechblog.com/automated-canary-analysis-at-netflix-with-kayenta-3260bc7acc69

  • Documentation: Detailed documentation was maintained for each microservice, including API specifications, architectural decisions, and operational guidelines. This documentation was essential for onboarding new team members and ensuring consistency across the organization.
  • Flexibility: The architecture was designed to be adaptable, allowing Netflix to quickly respond to changing requirements and scale services as needed. Continuous integration and continuous deployment (CI/CD) practices enabled rapid iteration and deployment.

“Flexibility requires an open mind and a welcoming of new alternatives.” — Deborah Day

By adopting a combination of architecture patterns and leveraging a robust technology stack, Netflix successfully transformed their monolithic application into a scalable, resilient, and rapidly deployable microservices architecture. This transition not only addressed their immediate challenges but also positioned them for future growth and innovation.

The approach a solution architect takes can significantly impact the success of a project. By following a structured process that starts with understanding the problem, moving through high-level and low-level design, and incorporating feedback and flexibility, a solution architect can create robust, scalable, and effective solutions. This methodology not only addresses immediate business needs but also lays a strong foundation for future growth and adaptability. The case of Netflix demonstrates how applying these principles can lead to successful, scalable, and resilient architectures that support business objectives and user demands.

Stackademic 🎓

Thank you for reading until the end. Before you go:

Kubernetes 101: Deploying & Scaling a Microservice Application

Leave a reply

Clone the Git Repository

First, clone the Git repository that contains the pre-made descriptors for the Robot Shop application.

cd ~/
git clone https://github.com/instana/robot-shop.git

Thanks to Instana for providing the Robot Shop application!

Create a Namespace

Since the Robot Shop application consists of multiple components, it’s a good practice to create a separate namespace for the application. This isolates the resources and makes management easier.

kubectl create namespace robot-shop

Deploy the Application

Deploy the application to the Kubernetes cluster using the provided descriptors.

kubectl -n robot-shop create -f ~/robot-shop/K8s/descriptors/

Check the Status of the Application’s Pods

To ensure the deployment was successful, check the status of the application’s pods.

kubectl get pods -n robot-shop

Access the Robot Shop Application

You should be able to reach the Robot Shop application from your browser using the Kubernetes master node’s public IP.

http://<kube_master_public_ip>:30080

Scale Up the MongoDB Deployment

To ensure high availability and reliability, scale up the MongoDB deployment to two replicas instead of just one.

Edit the Deployment Descriptor

Edit the MongoDB deployment descriptor.

kubectl edit deployment mongodb -n robot-shop

In the YAML file that opens, locate the spec: section and find the line that says replicas: 1. Change this value to replicas: 2.

spec:
  replicas: 2

Save and exit the editor.

Check the Status of the Deployment

Verify that the MongoDB deployment has scaled up to two replicas.

kubectl get deployment mongodb -n robot-shop

After a few moments, you should see the number of available replicas is 2.

Add a New Replica Set Member

To further ensure data redundancy, add the new MongoDB replica to the replica set.

Execute MongoDB Shell

Use kubectl exec to open a MongoDB shell session in one of the MongoDB pods.

kubectl exec -it mongodb-5969679ff7-nkgpq -n robot-shop -- mongo

Replace <mongodb-pod-name 1> with the name of one of the MongoDB pods.

Add the New Replica Set Member

In the MongoDB shell, run the following command to add the new member to the replica set.

Check the status of the replica set.

rs.status()

Add the other MongoDB pod to the replica set.

rs.add("mongodb-5969679ff7-w5kpg:27017")

By following these steps, you have successfully deployed the Robot Shop application, scaled up the MongoDB deployment for high availability, and added a new replica set member to ensure data redundancy. This setup helps in maintaining a reliable and robust application environment.

Stackademic 🎓

Thank you for reading until the end. Before you go: