Designing the right architecture for a problem statement is more of an art than a science because it depends a lot on the understanding of the problem statement, the context, and where we think it will expand next. The most important thing about any architecture is how adaptable it is in the face of changing requirements of business and scale. Below are my experiences of how different architecture styles, principles, and methodology come together to form an architecture that is evolution-ready.

What is bad architecture and how to recognize it?

For the sake of velocity of development, developers often put in bad code, which ultimately leads to what we…

Agile methodology is put to practice using the shift-left-testing. Shift left testing paradigm suggests “Test early, Test often”. Test Driven Development is the approach towards development puts this thought process into action in Agile teams.

The onus of quality is not of the Qualify Assurance team. The QA Team is only a gatekeeper of quality to ensure only quality product goes out of the door. The onus of quality belongs to developers. Quality has several shades — Functional correctness & completeness, Non-Functional performance compliance, readability, maintainability, extendibility etc., to mention a few.

In this article, I am going to share…

This article gives a developer’s perspective of using Splunk, get quickly acquainted with Splunk, understand SQL (Splunk Query Language) with some tips and tricks along the way to write simple, readable and optimized queries

Developer Lab setup for practice

You can download the trial edition of Splunk or the Splunk cloud to get started. Anything which can give results using regex can be used as an input to Splunk. For example: CSV files, JSON, log files. Data can be uploaded to Splunk, or it can be streamed to Splunk

Uploading Data

In enterprise set up, would have been already loaded from the different system logs using Splunk…

This article gives high level overview of splunk — Why splunk? What is splunk? and the Architecture of splunk.

Any Question, Any Data, One Splunk

Splunk is not a mere log aggregator, it has the capability to do data-analytics in almost real-time by cleaning, segregating, extracting, classifying stream of machine generated data. It helps in visualization & analysis of the data in the form of charts & statistics by creating dashboards.

It thereby helps pro-actively monitor systems, trouble shoot failure, store & retrieve data for later use, investigate particular outcome, identify operational loop holes, monitor business metrics, derive business insights, analyse system performance, security threats, maintain…

How can I derive value from the data that is stored in MongoDB?

The ultimate objective of storing all the data is to derive business insights, find new business opportunities, reduce operational costs or simply optimization of some form — which means data-analytics.

Analytics of big-data

Typical business needs are mostly aggregation of the data in meaningful way. There are 3 approaches to achieve this in MongoDB.

  1. Find command — Simple requirement, extremely limited in scope.
  2. Aggregation Pipeline — This satisfies 90% of analytic needs in MongoDB. Together with its power, speed and simplicity it is the go to choice for most developers.

Sharding is Mongo’s answer for scaling. It is a way of distributing both work and data. More work over more data. And it is required for big-data problems — really really big data. So it is important to understand

  1. What is sharding?
  2. How is it done?
  3. Pitfalls & how to design your data distribution strategy?
  4. When it should be done?
  5. When is it too late?

Horizontal scaling (scaling-out)

It achieves the scaling requirements by scaling out, i.e. by adding more nodes (machines). This is also called horizontal scaling. Horizontal scaling is cheaper than vertical scaling. …

Purpose & Layout for the 5 part series

When I started on MongoDB, I found there is not a good step-by-step guide that I found, to get started, understand what MongoDB offers, understand the design considerations when migrating from RDBMS to NoSql and then use MongoDB so as to unleash its full potential. This is a developer’s view of using MongoDB and understand the basics.

MongoDB: Step-1: An overview
MongoDB: Step-2: Setting up the server
MongoDB: Step-3: Mongo Client Commands
MongoDB: Step-4: Designing the schema
MongoDB: Step-5: Indexes and Query Performance Optimizations —

Step1 — An overview

This covers the evolution of databases, the data explosion (big data)…

Step2 — Server & Client set up

This covers the setting up of the server on the cloud, the options for mongo client, typical mongo replica setup.

MongoDB server (on local)

MongoDB is open-source and the community edition of the server is available for free download.

Download the mongo server as per your OS and follow the instructions as mentioned in the below link. You will be surprised by the size of MongoDB package and the ease of installation if you have gone through some of the RDBMS installations in the past.

--dbpath option points to your database directory. Create one folder for this, say it it “c:\data\db”

Run the below…

Step3-Mongo Client commands

Introduces you to Mongo-Shell and the most commonly used Mongo commands.

Mongo Shell

Mongo Shell is a javascript interpreter that interactively connects to the mongo server and helps you interactively talk to your mongo server. i.e. the command on it does not finish but you can write the next command which can use the scripting set before it.

The shell is therefore used to

  1. Connect to MongoDB and fire the DML queries
  2. Do administrative tasks

Connecting without specifying a database

If you connect to mongo without specifying the database your will land in the test database

Customizing the shell prompt

You can…

Step-4: Designing the schema

We will touch upon the principles that can be used for defining & determining the schema design for any project. Once you apply this you will arrive upon a fluid schema upon which you may then apply design patterns to arrive at the final schema.

The secret to what schema you will arrive at depends on your application & below are some of the key considerations that will help you get there.

  1. Does your application do more reads than writes?

2. What data needs to be together when reading from the database?

3. What performance considerations are there?

4. How…

Sarada Sastri

Java Architect | MongoDB | Oracle DB| Application Performance Tuning | Design Thinking |

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store