Designing the right architecture for a problem statement is more of an art than a science because it depends a lot on the understanding of the problem statement, the context, and where we think it will expand next. The most important thing about any architecture is how adaptable it is in the face of changing requirements of business and scale. Below are my experiences of how different architecture styles, principles, and methodology come together to form an architecture that is evolution-ready.
For the sake of velocity of development, developers often put in bad code, which ultimately leads to what we…
Agile methodology is put to practice using the shift-left-testing. Shift left testing paradigm suggests “Test early, Test often”. Test Driven Development is the approach towards development puts this thought process into action in Agile teams.
The onus of quality is not of the Qualify Assurance team. The QA Team is only a gatekeeper of quality to ensure only quality product goes out of the door. The onus of quality belongs to developers. Quality has several shades — Functional correctness & completeness, Non-Functional performance compliance, readability, maintainability, extendibility etc., to mention a few.
In this article, I am going to share…
This article gives a developer’s perspective of using Splunk, get quickly acquainted with Splunk, understand SQL (Splunk Query Language) with some tips and tricks along the way to write simple, readable and optimized queries
You can download the trial edition of Splunk or the Splunk cloud to get started. Anything which can give results using regex can be used as an input to Splunk. For example: CSV files, JSON, log files. Data can be uploaded to Splunk, or it can be streamed to Splunk
In enterprise set up, would have been already loaded from the different system logs using Splunk…
This article gives high level overview of splunk — Why splunk? What is splunk? and the Architecture of splunk.
Splunk is not a mere log aggregator, it has the capability to do data-analytics in almost real-time by cleaning, segregating, extracting, classifying stream of machine generated data. It helps in visualization & analysis of the data in the form of charts & statistics by creating dashboards.
It thereby helps pro-actively monitor systems, trouble shoot failure, store & retrieve data for later use, investigate particular outcome, identify operational loop holes, monitor business metrics, derive business insights, analyse system performance, security threats, maintain…
How can I derive value from the data that is stored in MongoDB?
The ultimate objective of storing all the data is to derive business insights, find new business opportunities, reduce operational costs or simply optimization of some form — which means data-analytics.
Typical business needs are mostly aggregation of the data in meaningful way. There are 3 approaches to achieve this in MongoDB.
Sharding is Mongo’s answer for scaling. It is a way of distributing both work and data. More work over more data. And it is required for big-data problems — really really big data. So it is important to understand
It achieves the scaling requirements by scaling out, i.e. by adding more nodes (machines). This is also called horizontal scaling. Horizontal scaling is cheaper than vertical scaling. …
When I started on MongoDB, I found there is not a good step-by-step guide that I found, to get started, understand what MongoDB offers, understand the design considerations when migrating from RDBMS to NoSql and then use MongoDB so as to unleash its full potential. This is a developer’s view of using MongoDB and understand the basics.
MongoDB: Step-1: An overview
MongoDB: Step-2: Setting up the server
MongoDB: Step-3: Mongo Client Commands
MongoDB: Step-4: Designing the schema
MongoDB: Step-5: Indexes and Query Performance Optimizations —
Step1 — An overview
This covers the evolution of databases, the data explosion (big data)…
This covers the setting up of the server on the cloud, the options for mongo client, typical mongo replica setup.
MongoDB is open-source and the community edition of the server is available for free download.
Download the mongo server as per your OS and follow the instructions as mentioned in the below link. You will be surprised by the size of MongoDB package and the ease of installation if you have gone through some of the RDBMS installations in the past.
--dbpath option points to your database directory. Create one folder for this, say it it “c:\data\db”
Run the below…
Introduces you to Mongo-Shell and the most commonly used Mongo commands.
The shell is therefore used to
Connecting without specifying a database
If you connect to mongo without specifying the database your will land in the test database
Customizing the shell prompt
We will touch upon the principles that can be used for defining & determining the schema design for any project. Once you apply this you will arrive upon a fluid schema upon which you may then apply design patterns to arrive at the final schema.
The secret to what schema you will arrive at depends on your application & below are some of the key considerations that will help you get there.
2. What data needs to be together when reading from the database?
3. What performance considerations are there?