Skip to main content

Unibench

Development

7 Proven Ways to Reduce Application Latency

Every millisecond counts in today’s digital world. A slow-loading application doesn’t just frustrate users, it directly impacts revenue, retention and trust. But how can organizations actually reduce latency in their applications? Let’s look at some proven strategies to reduce application latency that top-performing teams rely on: 1. Optimize Database Queries– Use proper indexing to avoid full table scans.– Reduce “N+1” query problems by batching requests.– Consider read replicas for high-demand applications.– Implement caching layers like Redis or Memcached to minimize repetitive database calls. 2. Leverage Content Delivery Networks (CDNs)– CDNs cache content closer to users, cutting down geographical latency.– Assets such as images, scripts, and videos should always be served via CDN.– Popular options: Cloudflare, Akamai, AWS CloudFront. 3. Implement Caching Strategically– Cache at multiple levels: client-side, server-side, and edge.– Apply HTTP caching headers effectively.– Use in-memory caching for frequently accessed data. 4. Optimize Application Code– Refactor inefficient loops and algorithms.– Reduce dependencies and heavy third-party libraries.– Adopt asynchronous processing for I/O heavy operations. 5. Choose the Right Architecture– Microservices enable scaling of specific components instead of the entire app.– Serverless functions reduce idle resource usage.– Event-driven systems improve responsiveness under load. 6. Monitor and Profile Regularly– Tools like New Relic, Datadog or Prometheus identify bottlenecks in real-time.– Regular load testing ensures systems handle peak traffic.– A proactive monitoring culture prevents issues before users notice them. 7. Reduce Network Latency– Compress payloads with Gzip or Brotli.– Use HTTP/2 or HTTP/3 for faster transfers.– Keep APIs lightweight and avoid unnecessary data in responses. Latency isn’t solved by one silver bullet. It’s the result of multiple small, consistent optimizations across your database, infrastructure, and codebase. Faster applications deliver smoother user experiences, improve customer trust and set companies apart from competitors. In industries like fintech, e-commerce and SaaS, speed directly translates to growth.

Development

Choosing the Right Test Automation Tools; A Comparative Analysis

Selecting the right test automation tools is crucial for the success of your automation efforts. The effectiveness of your test automation strategy largely depends on choosing tools that align with your project’s needs, your team’s expertise, and your budget constraints. A well-chosen tool can enhance productivity, ensure comprehensive coverage, and integrate seamlessly into your existing workflows. Tool Categories Functional Testing Tools Functional testing tools are used to validate that the software behaves as expected under various conditions. Popular tools include; Selenium Selenium is a widely-used open-source tool for automating web applications. It supports multiple programming languages and browsers, making it highly versatile. Cypress Cypress is a newer tool known for its fast execution and ability to provide real-time browser interaction. It is designed specifically for JavaScript applications and provides a more modern approach to end-to-end testing. Playwright Playwright is a cross-browser automation library that supports modern web applications. It offers powerful features for testing across different browsers and devices, and is known for its reliability and speed. API Testing Tools API testing tools are essential for verifying that APIs function correctly and meet performance standards. Key tools include; Postman Postman is a popular tool for manual API testing and automation. It provides an intuitive interface for creating and running API requests and tests. RestAssured RestAssured is a Java library for testing RESTful APIs. It offers a fluent API for writing and executing tests and integrates well with other testing frameworks. PactumJS PactumJS is an open-source tool designed for contract testing of APIs. It supports testing REST and GraphQL APIs and is known for its ease of use and flexibility. Performance Testing Tools Performance testing tools are used to assess how the application performs under various load conditions. Common tools include; JMeter Apache JMeter is a powerful tool for load testing and performance measurement. It supports a wide range of protocols and offers extensive reporting capabilities. LoadRunner LoadRunner is a comprehensive performance testing tool that supports various types of performance tests, including load, stress, and endurance testing. Gatling Gatling is a modern performance testing tool known for its high performance and ease of use. It uses a Scala-based DSL for creating complex load tests. Criteria for Selection Ease of Use Consider how user-friendly the tool is. An intuitive interface and ease of setup can significantly impact productivity and reduce the learning curve for your team. Integration Assess whether the tool integrates well with your Continuous Integration/Continuous Deployment (CI/CD) pipeline. Seamless integration ensures that automated tests are executed efficiently as part of your development process. Community Support Evaluate the level of community or vendor support available. A strong support network can be crucial for troubleshooting issues and obtaining updates or enhancements. Scalability Determine if the tool can handle the complexity and size of your project. Scalability ensures that the tool can accommodate growing testing needs and adapt to evolving project requirements. Comparative Analysis Create a matrix to compare popular tools based on features, pricing, and usability. This matrix should provide a clear overview of how each tool stacks up against the criteria outlined above, helping you make an informed decision based on your specific needs and constraints.

Development

DevOps Best Practices for Continuous Integration and Continuous Delivery (CI/CD)

In the fast-paced world of software development, achieving both speed and quality is essential. DevOps, which combines development and operations, is a powerful approach to meet these goals. At the heart of DevOps are Continuous Integration (CI) and Continuous Delivery (CD), processes that streamline the development, testing, and deployment of software. Let’s dive deeper into some best practices for making the most out of CI/CD. What is CI/CD? Before we jump into best practices, let’s clarify what CI/CD is: Continuous Integration (CI): This involves developers frequently merging their code changes into a shared repository where automated builds and tests run. This helps catch and fix problems early. Continuous Delivery (CD): After the CI stage, code changes are automatically prepared for a release to production, ensuring that the software can be deployed at any time. Best Practices for CI/CD Keep a Single Source Repository Maintaining a centralized repository like Git is crucial. It ensures that all team members work with the same code base and makes tracking changes easier. Popular platforms include GitHub, GitLab, and Bitbucket. Having a single source of truth for your codebase helps prevent conflicts and confusion. It ensures that everyone on the team is on the same page and that all changes are tracked and documented. This setup also makes it easier to manage code reviews, integrate new features, and collaborate effectively. Automate Builds and Tests Automation is key in CI/CD. Every time code is committed, automated builds and tests should run. This ensures that the new code integrates well with the existing codebase and doesn’t introduce bugs. Automated builds and tests save time and reduce human error. By running tests automatically with every code change, you can catch bugs early when they are easier and cheaper to fix. This practice also helps maintain a high level of code quality and ensures that your application remains stable as new features are added. Commit Code Frequently Encourage developers to make small, incremental changes and commit them frequently. This practice makes it easier to identify which commit caused a problem if something goes wrong. Frequent commits lead to smaller changesets, which are easier to review and debug. This practice also reduces the risk of conflicts and makes it easier to integrate new features. By committing code regularly, developers can get quicker feedback on their changes, which helps in maintaining a steady development pace. Code Reviews and Pair Programming Before merging code into the main branch, have at least one other developer review it. Code reviews help catch potential issues and improve code quality. Pair programming, where two developers work together on the same code, can also be beneficial. Code reviews and pair programming foster collaboration and knowledge sharing. They help catch bugs and improve code quality by providing an additional set of eyes on the code. This practice also promotes best coding practices and helps junior developers learn from more experienced team members. Use a Staging Environment A staging environment that mirrors production allows you to test the software in a setting that’s as close to the real world as possible before releasing it to users. Testing in a staging environment helps catch issues that might not be apparent in a development environment. It ensures that the software works as expected under conditions that closely resemble the production environment. This practice reduces the risk of deploying faulty code to production and helps maintain a high level of user satisfaction. Monitor and Optimize Build Times Long build times can slow down development. Regularly monitor and optimize build times. This might involve improving test efficiency or upgrading infrastructure. Efficient build processes keep the development workflow smooth and fast. By minimizing build times, you reduce the feedback loop for developers, allowing them to iterate quickly and effectively. Optimizing build times also helps in maintaining the momentum of the development process and prevents bottlenecks. Implement Feature Toggles Feature toggles (or flags) allow you to deploy new features to production but keep them disabled until they’re ready for release. This enables continuous delivery without exposing unfinished features to users. Feature toggles provide flexibility in deploying new code. They allow you to merge code into the main branch without worrying about unfinished features being visible to users. This practice also makes it easier to test new features in production and roll them out gradually, reducing the risk of introducing bugs. Integrate Security Checks Security should be integrated into your CI/CD pipeline. Use tools like static code analysis and vulnerability scanners to detect security issues early in the development process. Integrating security checks into your CI/CD pipeline ensures that security is a continuous consideration, not an afterthought. Early detection of security vulnerabilities helps prevent potential breaches and ensures that your application remains secure. This practice also helps maintain compliance with security standards and regulations. Plan for Rollbacks Be prepared for failures. Implement strategies to quickly roll back to a previous stable state if a deployment causes issues. This might involve automated rollback scripts or maintaining a version history. Rollbacks provide a safety net in case something goes wrong during deployment. Having a rollback plan in place ensures that you can quickly revert to a stable state, minimizing downtime and disruption for users. This practice also builds confidence in the deployment process, knowing that you have a fallback option. Continuous Monitoring and Feedback Once the software is deployed, monitor its performance and gather user feedback. Use this information to make continuous improvements. Monitoring tools can alert you to issues in real-time, allowing for quick responses. Continuous monitoring helps ensure that your software performs well in production. By gathering user feedback and monitoring performance metrics, you can identify and address issues quickly. This practice helps maintain a high level of user satisfaction and provides valuable insights for future improvements. Adopting DevOps practices for Continuous Integration and Continuous Delivery can significantly improve the speed and quality of software development. By automating builds and tests, encouraging frequent commits, and maintaining robust testing environments, you can streamline your development process

Development

Introduction to Test Automation, When and Why to Automate

What is Test Automation Test automation involves the use of specialized tools and scripts to execute tests automatically rather than manually. This approach is particularly effective for repetitive and time-consuming tasks, helping to improve both test coverage and efficiency. Automation allows for consistent execution of tests, rapid feedback, and more comprehensive validation of the software’s functionality. When to Automate Not all tests should be automated. Automation is most beneficial in the following scenarios, Regression Testing This involves repeatedly running the same set of tests to verify that recent changes or updates to the application have not introduced new defects. Automated regression tests help ensure that existing functionality remains intact after code changes. Performance Testing Automation is crucial for conducting load, stress, and scalability tests. These tests simulate the behavior of hundreds or thousands of users to evaluate how the application performs under various conditions. Automation enables the execution of these tests efficiently and accurately. Smoke Testing Smoke testing involves running a suite of basic tests to verify that the core functionality of the application is working as expected. Automated smoke tests help quickly identify critical issues early in the development cycle, ensuring that the build is stable for further testing. Data-Driven Testing This type of testing involves running the same test case multiple times with different sets of input data. Automation facilitates data-driven testing by enabling the execution of tests with various data inputs efficiently, which would be tedious and time-consuming to perform manually. Benefits of Automation Efficiency Automation can execute tests significantly faster than manual testing, especially for large test suites. Automated tests can be scheduled to run overnight or during off-hours, allowing for continuous testing and faster feedback. Consistency Automated tests are executed in a consistent manner every time they run. This reduces the risk of human error associated with manual testing, ensuring that the tests are performed uniformly and accurately across different test cycles. Reusability Once created, automated test scripts can be reused across multiple projects and test cycles. This reusability reduces the need for repetitive test creation and enables efficient testing of similar functionalities in different versions or applications. Challenges of Automation Initial Setup Costs Setting up an automation framework and developing the initial set of test scripts can be time-consuming and costly. This involves selecting appropriate tools, designing the test architecture, and writing the test cases, which can require significant investment in terms of both time and resources. Maintenance Automated tests require ongoing maintenance to ensure they remain effective and relevant. As the application evolves, test scripts may need to be updated to reflect changes in functionality or user interfaces. This ongoing maintenance can be resource-intensive and needs to be managed to keep the automation framework aligned with the current state of the application.

Development

The Shift-Left Approach, Integrating QA Early in the Development Process

What is Shift-Left Testing Shift-Left is a testing approach that involves integrating QA activities early in the Software Development Life Cycle (SDLC), often during the requirements and design phases. The core idea is to catch defects as early as possible, which helps in reducing the cost and effort required to fix them later. By addressing potential issues at the early stages, teams can prevent them from becoming entrenched in the codebase, leading to a more efficient development process. Why Shift-Left is Crucial Traditional testing approaches, which occur after development is completed, often fall short in preventing costly defects. Issues discovered late in the development cycle can be expensive and time-consuming to fix. By shifting QA activities to earlier stages, teams can identify and resolve problems before they become embedded in the codebase. This approach not only enhances the quality of the software but also accelerates delivery times, ultimately benefiting the project’s overall success. Implementation Strategies Involving QA in Requirements Gathering To ensure that requirements are clear, testable, and aligned with business objectives, it is essential to include QA professionals in the early stages of requirements gathering. Their involvement helps in defining acceptance criteria and identifying potential issues or ambiguities in the requirements. This early involvement ensures that the requirements are feasible and that the testing process is well-prepared from the start. Continuous Integration and Testing Implementing Continuous Integration (CI) pipelines is a key strategy in the Shift-Left approach. CI pipelines automatically run tests every time code is committed, which helps in detecting and addressing issues immediately. This continuous feedback loop allows developers to catch and fix defects as soon as they are introduced, reducing the chances of defects accumulating over time. Test-Driven Development (TDD) Test-Driven Development encourages developers to write test cases before writing the code itself. This approach ensures that the code is developed to meet the test criteria from the outset. By focusing on the tests first, developers are guided by clear requirements and expectations, which leads to better-designed and more reliable code. Benefits Reduced Defects By identifying and addressing defects early in the development process, the overall defect count is reduced. This early detection leads to higher-quality software and minimizes the need for extensive bug fixes later in the project. Faster Time-to-Market Early testing and defect resolution reduce the need for major corrections and extensive testing later in the project. As a result, teams can deliver software more quickly and efficiently, improving time-to-market and meeting project deadlines more effectively. Improved Collaboration The Shift-Left approach fosters closer collaboration between developers, testers, and business analysts. By involving QA professionals from the beginning, communication and understanding among team members are enhanced, leading to more cohesive and coordinated efforts throughout the development process.

Development

Manual Testing, Laying the Groundwork for Effective Quality Assurance

Manual Testing Overview Manual testing is the process of executing test cases as per user functionality, without the help of automation tools. It is a basic testing concept in which the tester behaves as a real end-user and is expected to run through every feature of an application. This approach provides significant usability, functional, and general user experience insights. Why Manual Testing Matters Despite the rise of automation, manual testing remains crucial for scenarios that require human intuition. This includes usability assessments, exploratory testing of newly developed features, and verification of complex user interactions where automation scripts may not yet be established. Manual testing helps in identifying issues that automated tests might overlook, particularly those involving subjective user experiences and interactions. Key Components Test Case Design Crafting effective test cases is essential for thorough manual testing. A well-designed test case should be clear, concise, and inclusive of various scenarios. It typically comprises: Test Case ID – A unique identifier assigned to each test case for easy reference and traceability. Test Description – A summary outlining what the test case aims to verify, including the functionality or feature being tested. Preconditions – Conditions that must be met before the test can be executed, ensuring the test environment is correctly set up and any necessary data or configurations are in place. Test Steps – Detailed, sequential instructions on how to execute the test, guiding the tester through the process to ensure consistency and accuracy. Expected Result – The anticipated outcome if the application behaves as intended, serving as the benchmark for determining test success. Actual Result – The outcome observed during test execution. This is documented only when there is a deviation from the expected result, indicating a potential issue or bug. Documenting the actual result helps in tracking discrepancies and guiding further debugging or development efforts. Status – A final indication of whether the test passed or failed, based on the comparison between the expected and actual results. Common statuses include Pass, Fail, and Incomplete. Defect Reporting When a discrepancy between the expected and actual results is observed, it is crucial to document the defect thoroughly. This includes: Defect ID – A unique identifier for tracking the defect. Summary – A brief description of the defect. Steps to Reproduce – Detailed steps to replicate the issue, aiding developers in understanding and fixing the problem. Severity and Priority – Assessment of the defect’s impact on the application and its urgency for resolution. Attachments – Screenshots, logs, or other evidence supporting the defect report. Test Execution and Management Effective test execution involves managing test cycles, tracking progress, and ensuring timely completion of tests. This includes: Test Execution – Running the test cases as per the defined steps, noting any deviations from the expected results. Test Cycle Management – Organizing test cases into cycles based on features, releases, or testing phases to ensure comprehensive coverage. Test Reporting – Summarizing test results, including pass/fail rates, defect metrics, and overall test coverage. Reports provide insights into the quality of the application and help in decision-making. By incorporating these components into the manual testing process, teams can ensure a thorough evaluation of the software, identify potential issues early, and enhance overall quality assurance efforts.

Development

Streamlining API Automation with Node.js
and Cucumber: A Comprehensive Guide

Author: Sandeep Perera Date: Apr 18, 2024 In the ever-evolving landscape of software development, automation testing has become an indispensable component of the development lifecycle. APIs (Application Programming Interfaces) serve as the backbone of modern applications, enabling seamless communication between different software systems. Consequently, ensuring the reliability and functionality of APIs through automation testing is crucial. In this article, we’ll explore how to streamline API automation using Node.js and Cucumber, a powerful combination for creating robust and maintainable automated tests. Understanding API Automation API automation involves testing the functionality, reliability, performance, and security of APIs without manual intervention. It ensures that APIs behave as expected under various conditions and scenarios. Automated API testing offers several benefits, including: Efficiency: Automated tests can be executed quickly and repeatedly, saving time and effort compared to manual testing. Accuracy: Automation eliminates human errors, providing more reliable test results. Coverage: Automated tests can cover a wide range of use cases, including edge cases and negative scenarios, enhancing test coverage. Regression Testing: Automation enables continuous testing, facilitating early detection of regressions and preventing software defects. Introducing Node.js and Cucumber Node.js has gained widespread adoption in the development community for building fast, scalable, and efficient server-side applications. Its non-blocking, event-driven architecture makes it well-suited for handling asynchronous operations, such as network communication. Additionally, Node.js boasts a rich ecosystem of libraries and frameworks, making it an ideal choice for API automation. Cucumber is a popular open-source tool for behavior-driven development (BDD). It allows developers and testers to write test scenarios in plain text using the Gherkin syntax, which is easily understandable by non-technical stakeholders. Cucumber promotes collaboration between different roles in the development process and encourages the creation of living documentation. Setting Up the Environment Before diving into API automation with Node.js and Cucumber, you need to set up your development environment. Ensure you have Node.js and npm (Node Package Manager) installed on your machine. You can create a new Node.js project by running: Next, install the necessary dependencies: Writing API Tests with Cucumber Cucumber follows a simple, human-readable syntax for describing test scenarios. A typical Cucumber feature file consists of scenarios written in Gherkin syntax, which describe the behavior of the system under test. Here’s an example feature file for testing a RESTful API: Each step in the scenario is associated with a step definition, which is implemented in JavaScript. Let’s implement the step definitions using Axios, a popular HTTP client for Node.js: Running Tests To execute the Cucumber tests, run the following command in your terminal: Cucumber will parse the feature files and execute the associated step definitions. You’ll see the test results in the terminal output, indicating whether each scenario passed or failed. Conclusion Node.js and Cucumber offer a powerful combination for automating API tests with ease. By leveraging the expressive syntax of Cucumber and the versatility of Node.js, you can create robust, maintainable tests that accurately verify the behavior of your APIs. Whether you’re testing RESTful APIs, GraphQL endpoints, or SOAP services, Node.js and Cucumber provide a flexible and efficient solution for API automation. Start streamlining your API testing workflow today!

Development

New Java language features worth checking out!
(2024) — Part 1

Author: Kokila Widyaratna Date: 02/04/2024 Photo by bluebay2014 @ Getty Images I started using Java in 2017 with version 8 where features like lambda expressions, stream API, default method, optional wrapper..etc, were getting a huge hype. I’ve been incorporating almost all those features during my day-to-day development work. Though I have been upgrading from Java 8 to 11 and then to 17, I noticed that I have been using the same old features. Of course, we are not bumping these Java versions only to get new language features but for performance and security improvements as well. But with the intention of getting to know new features in the language, I started doing a little bit of exploring and found the following talk by Venkat Subramaniam which I recommend to every Java developer. This post is inspired by this video. Feature 1 — Switch Expression (Java 14) Before going into this feature, let us look at the below code snippet and guess the console output. The console output is; Kudos if you guessed it correctly but this is a mistake which wasted a lot of my time during my early years. If you look at the first case, you can see that the break statement is missing and this is why the above code snippet is behaving in that manner. Now let’s see how the switch expression feature which was introduced as a preview in Java12 and then a standard feature in Java14, minimizes this error from happening. And the output is; console output of the example code snippet of the new switch expression feature As you can see, a break statement is not required in the switch expression. The ability to handle multiple cases is also a feature of the switch expression. An example snippet and console output is given below. Please refer to line #6. console output of the example code snippet for multiple case handling The yield keyword is also something which was introduced with this switch expression feature where we can use it to exit a switch expression by returning a value that becomes the value of the switch expression. This means we can assign the value of a switch expression to a variable. Refer to line #9 and the console output below as an example. console output of the example code snippet for the ‘yield’ keyword That is all for Part 1 of this series. In Part 2, I am planning to discuss the Java Records feature which was introduced as a preview in Java 14 and then became a standard feature in Java 16. Hope this can help! Share your thoughts too.

Development

How Git Works — Part 2: Commits, Branches, and Merge

In the first part of this series, we understood the fundamental building blocks of the git storage model. Since we have these blocks we can examine how Git uses these building blocks to do the real version controlling. Now you know commit will always refer to the tree object that represents the root of the repository directory which will refer to all of its sub-tree objects and blobs. Every time when you commit something a new commit object will be created and the parent of that commit will be the last commit that you start work in. The missing piece is to track the last commit that the root directory is in when you go to the repository directory which means the last saved state in git. That is done through the HEAD file in the .git folder. Let’s see the content of it (This repo contains one file and one commit) ref: refs/heads/master Wait what it is not a commit object. That’s where the concept of branches comes into the picture. If this file refers to the latest commit directly then the commit tree will actually be a linked list. The goal of the git was not only to allow a simple version control system. It is also to make it easy to collaborate and contribute to the repository by many people. Hence it introduces the concept of branches. The branch contains the last commit of that branch and the HEAD file refers to the current branch. So as in the content of the HEAD file let’s examine the refs/heads/master file inside the .git folder 47b45e9fa099bc72b2a919d8ee97a68ca82c60fd Now you see it is a hash value that represents a commit object. Git will create a default branch called master when you use the git init command. Let’s create a new branch and examine the .git directory git branch <branch_name>git checkout <branch_name> or use this single command to create and checkout in one step git checkout -b <branch_name> The content will be the last commit of the current branch when you create and switch to the new branch The main advantage of having branches is you can keep a stable branch and create feature branches for feature development. People can work independently without worrying too much about others’ changes. But finally, you need your changes to be in the main branch. That is where the concept of merge comes into the picture Git is able to interpret the difference between two commits (The output of the git diff command). and able to apply these diffs as a Commit. This is what actually happens in merging. There are many merging algorithms available in Git let’s consider the most used approaches and explain them. Let’s consider the master branch as the branch that we need our new changes to get merged. Fast forward merge Let’s consider a scenario where you create a new branch from the master and do some new changes and commit to that new branch. Now you need these changes to be available at the master branch and the master branch has no mere new commit after you create the new branch. So now the easiest way to merge these branches is by just simply changing the master branch commit to the new branch commit. No need for diffing or any new commit, not clear? let’s visualize Three-way merge What if there is a commit in the master branch after that you create your new branch? Now you can’t directly move the master commit to the new branch commit as in the previous case. The solution is to introduce a new commit. When creating this new commit git has to consider three points to calculate the diffs and apply. The commit that the new branch created, the latest commit of the master branch, and the latest commit of the new branch. That is why it is called a three-way merge. The new Commit will have two parents. This process might not be this simple if these changes are overlapping. Then in this merge commit you will also have to resolve these conflicts. Git is able to help you resolve these conflicts and modern IDEs make this process more simple. If you are interested in learning in-depth how git diffing patching and conflict resolving happen please watch this series. The git merge does not change the git history. A powerful feature that allows you to change the git history is the rebase. Let’s visualize the simplest example to see what happens when you use rebase Here the changes in the new branch are applied to the master branch as new commits. The commits have different hash values. It also has an interactive mode which is more powerful and allows us to do many operations like pick, squash, revert and etc. I will not delve deep into git rebase in this post, but it is a very powerful tool. There are plenty of resources out there that dig deeper into these subjects, but my goal was not to go that deep because, in the end, you might not use those git-pluming commands daily basis. But the power of having the understanding at a high level is very important and can help you in cases where you are stuck. From all these simple explanations the point that I want to elaborate on is that all git commands that you use on a daily basis are ways to play with this commit history. Now you can eliminate many myths. Deleting a branch in Git does not delete the commits immediately, hence don’t need to panic git reflog command can use to recover from these situations, same for git reset don’t fear using them. Just commit your changes and you can play with these commits if it was not as expected in the first place There is also a garbage collection process available in Git that will remove objects which are not referenced by any branches. It also does compression to reduce storage. These policies

Development Technology

How Git works- Part I – Object Storage

If you are a software developer, the likelihood that you use Git as your version control system is very high. Additionally, there is a good chance that you may encounter a circumstance where the commit history has been messed up at least once. What do you normally do in such situations? I guess you find some commands on Stack Overflow and, depending on the luck you have that day, you might resolve the problem quickly. Have you ever attempted to understand how Git functions? If the answer is yes, you may also be aware of the strength and solace it offers. However, if the response is negative, you might never consider it necessary. First of all, you must have adequate knowledge of Git’s operations. Let’s start by creating a local Git repository. Create an empty directory called “my_repo” and navigate inside it using the terminal. Run the command git status. It will indicate that it is not a Git repository (fair enough). So, how can we make a directory a Git repository? It’s simple! Run the command git init. Then you will notice that a new hidden directory called “.git” has been created, which stores all the magic that Git creates. Now, let’s see what is inside this directory. (I have used the tree /f .git command in Windows to visualize the tree) Content of the initial .git folder: Let’s add a file called “file1.txt” to the “my_repo” directory. Add simple text content (If you need to get the same result as shown, add a single line “first line of content” to the file), and run the command git status. You will see the following output: Git will not track anything automatically. You can verify it by looking at the “.git” directory. The above output explains how to track this file. Now let’s run git add file1.txt. Then, check the “.git” directory’s content. Two new files and one directory are now available in the “.git” folder. Let’s understand what these are. Git is a version control system, so it is required to store all the files that we track, version them, and provide a way to access these versions. The “objects” directory is used by Git for this purpose. Git has four types of objects: blobs, trees, commits, and annotated tags. Blobs: All the files (not directories) are stored as blobs. These are generated when you stage changes with the git add command. Hence, the file name “faed594a40f2d0aa7e45c5f26138cc24316565” is a blob that contains the content of “file1.txt”. But why is it in a directory called “d4”? The file name is also strange. This name is a hash generated by Git using the content of the file. Git provides a command by which you can generate this hash. Now you see “d4” is the first two letters of the hash, and “faed594a40f2d0aa7e45c5f26138cc24316565” is the remaining part. This directory works like an index. All the data with hash values starting with “d4” will end up in this directory, which increases the performance when searching for them. These files are in binary format. But Git provides commands to identify the type and content of these objects. Before understanding what this index file is, let’s add a directory called “sub” and copy “file1.txt” to this directory, renaming it to “file2.txt”. Then use the command git add . to track these changes. Now check the “.git/objects” directory. Did you notice that nothing new was added? It’s because “file2.txt” has the same content, so there’s no need to duplicate the same hash. A single blob but two files, I know that you are guessing about what is to be in the index file now. Given that it is also a binary file, we need the help of Git to view it. That’s what you call indexing or staging in Git. Every time you make a change in a file and stage it, a new blob will be created, and the index file keeps track of these changes. Even a single letter change in a file that is staged will create a new blob because the hash is changing, and the file will refer to the new blob hash. However, this is not enough for version controlling the files. That’s where commits come into play. Let’s commit the changes. Let’s see what has been added to the “.git/objects” directory. Before examining them, let’s think about what parts are missing in order to have proper version control. Remember that blobs do not contain any information about their file names or the directory in which they exist. During the staging phase, we have this data in the index file, and we need a similar file that contains this data. Since every file belongs to a directory, an obvious solution is to introduce an object that represents a directory and has references to its files and subdirectories. That’s what Git does by introducing an object type called a tree. In our case, we have two directories at the root of the repository, and the subdirectory is called “sub”, which means we have two tree objects. We can find and examine the two tree objects using the “git cat-file” command. Now, think about what the missing part is by asking the question “What version are we controlling?” The answer is the repository directory, which in our case is the “my_repo” directory. We already have a tree object for this directory. The only part required is a reference to it, which is the commit object. The commit object also contains details about the author and the timestamp. This is a special commit because it is the first commit. From the second commit onwards it will have pointers to its parent commits. Now we know the fundamental components of the Git storage model. However, branching, merging, garbage collection, and the remote repository are still lacking. These components are what make the true version control system complete. Let’s examine these ideas in more detail In the Part 2 of this blog post.

  • 1
  • 2

Contact Us

Let’s talk about building something great together.
Reach out—we’re here to answer questions, explore ideas, or start a new partnership.

Contact Us Phone