An Open Source OpenTelemetry APM | SigNoz

OpenTelemetry is a Cloud Native Computing Foundation(CNCF) project aimed at standardizing the way we instrument applications for generating telemetry data(logs, metrics, and traces). However, OpenTelemetry does not provide storage and visualization for the collected telemetry data.

APM stands for Application Performance Monitoring or Application Performance Management. APM tools help engineering teams effectively monitor their applications by monitoring key metrics for application performance. With time, the way applications are built and deployed has changed. With containerisation technologies, software systems have become distributed and are more dynamic than ever in production environments.

APM tools have also evolved to newer ways of reporting metrics for container-based applications. For newer age architectures like microservices and serverless, it’s difficult for engineering teams to have a central overview of how their applications are performing. But monitoring technology also evolved and gave birth to distributed tracing. With distributed tracing, you can trace user requests across services and protocols.

But setting up a robust monitoring and observability stack is challenging in distributed applications. The first step for setting up observability is to instrument your applications for generating telemetry data. OpenTelemetry provides a consistent instrumentation layer for your entire application stack, including open source frameworks and libraries. Let’s learn more about OpenTelemetry.

What is OpenTelemetry?

OpenTelemetry is an open-source collection of tools, APIs, and SDKs that aims to standardize the way we generate and collect telemetry data. It follows a specification-driven development. The OpenTelemetry specification has design and implementation guidelines for how the instrumentation libraries should be implemented. In addition, it provides client libraries in all the major programming languages which follow the specification.

OpenTelemetry was formed after the merger of two open-source projects - OpenCensus and OpenTracing in 2019. Since then, it has been the go-to open-source standard for instrumenting cloud-native applications.

The specification is designed into distinct types of telemetry known as signals. Presently, OpenTelemetry has specifications for these three signals:

• Logs
• Metrics and
• Traces

Together these three signals form the three pillars of observability. OpenTelemetry is the bedrock for setting up an observability framework. The application code is instrumented using OpenTelemetry client libraries, which enables the generation of telemetry data. Once the telemetry data is generated and collected, OpenTelemetry needs a backend analysis tool to which it can send the data.

OpenTelemetry can send to multiple backends. You can use different backends for each signal in OpenTelemetry. But managing different tools is not recommended. Engineering teams also need to correlate all signals for effective analysis. And that’s where an APM tool is needed.

Things to keep in mind while choosing an OpenTelemetry APM

Below is the list of factors that should be taken into consideration before selecting an OpenTelemetry APM:

• Support for all distinct signals of OpenTelemetry
  Currently, OpenTelemetry collects telemetry data in three distinct signals, namely, logs, metrics, and traces. Setting up a robust observability framework requires the use of all three signals. An OpenTelemetry APM should be able to ingest and visualize all three signals. Moreover, the frontend of the OpenTelemetry APM should also provide features to easily correlate the signals.

• Native support for OpenTelemetry semantic conventions
  In OpenTelemetry, every component of a distributed system is defined as an attribute. The attribute is nothing but a key-value pair. These attributes are defined by the OpenTelemetry specification as OpenTelemetry semantic conventions. For example, here is a glimpse of how HTTP conventions look like:

  Attribute	Description	Example
  http.method	HTTP request method	GET; POST; HEAD
  http.target	The full request target as passed in an HTTP request line or equivalent	/blog/june/
  http.scheme	The URI scheme that identifies the used protocol	http; https

  An OpenTelemetry APM should have native support to store data with OpenTelemetry semantic conventions. Existing observability vendors usually transform the data collected using OpenTelemetry semantic conventions into their propriety formats. But OpenTelemetry has a huge list of semantic conventions which might not be fully utilized in such scenarios.

• Should allow aggregates on trace data
  Running aggregates on trace data enables you to create service-centric views. OpenTelemetry also provides you the ability to create custom tags. Combined with custom tags and aggregated trace data gives you a powerful magnifying glass to surface performance issues in your services. For example, you can get the error rate and 99th percentile latency of customer_type: gold or deployment_version: v2 or external_call: paypal

• Open Source
  OpenTelemetry is an open source standard with a huge community backing. It is testimonial to the fact that community-driven projects can solve large complex engineering problems. It is not necessary for the OpenTelemetry APM to be open source.

  But having an open-source OpenTelemetry APM can enable you to have a full-stack open-source solution. Open-source solutions have more flexibility, and if you self-host, you don’t need to worry about things like data privacy.

Most SaaS APMs now claim to be 100% compatible with OpenTelemetry. But it’s difficult to move away from legacy systems. A solution built natively for OpenTelemetry can be a good choice for OpenTelemetry APM. And that’s where SigNoz comes into the picture.

SigNoz - An APM built natively for OpenTelemetry

SigNoz is a full-stack open source APM built natively to support OpenTelemetry. At SigNoz, we believe that OpenTelemetry is going to be the world standard for instrumenting cloud-native applications.

SigNoz supports OpenTelemetry semantic conventions and provides visualization for all three distinct types of signals supported by OpenTelemetry.

The steps to send telemetry data to SigNoz involves:

• Instrument application code with language-specific OpenTelemetry libraries
• Configure OpenTelemetry Exporters to send data to SigNoz
• Visualize and analyze telemetry data using SigNoz dashboards

Here’s a picture depicting how OpenTelemetry fits within an application and SigNoz.

SigNoz cloud is the easiest way to run SigNoz. You can sign up here for a free account and get 30 days of unlimited access to all features.

You can also install and self-host SigNoz yourself. It can be installed on macOS or Linux computers in just three steps by using a simple install script.

The install script automatically installs Docker Engine on Linux. However, on macOS, you must manually install Docker Engine before running the install script.

git clone -b main <https://github.com/SigNoz/signoz.git>
cd signoz/deploy/
./install.sh

You can visit our documentation for more installation option.

Once your application is instrumented with OpenTelemetry client libraries, the data can be sent to SigNoz by specifying a specific port on the machine where SigNoz is installed.

You can then use Signoz to monitor application metrics with out-of-box charts and visualization.

The tracing signal from OpenTelemetry instrumentation helps you correlate events across services. With SigNoz, you can visualize your tracing data using Flamegraphs and Gantt charts. It shows you a complete breakdown of the request along with every bit of data collected with OpenTelemetry semantic conventions.

SigNoz also lets you run aggregates on your tracing data. Running aggregates on tracing data enables you to create service-centric views, providing insights to debug applications at the service level. It also makes sense for engineering teams as they own specific microservices.

You can check out the SigNoz GitHub repo here:

Best Practices for OpenTelemetry Instrumentation

To get the most out of OpenTelemetry APM, consider these best practices:

1. Automatic vs. manual instrumentation:
   • Use auto-instrumentation libraries when available for quick setup
   • Supplement with manual instrumentation for custom business logic
2. Sampling strategies for optimal performance:
   • Implement tail-based sampling for production environments
   • Use head-based sampling for development and testing
3. Handling sensitive data in telemetry:
   • Implement data scrubbing in your instrumentation code
   • Use the OpenTelemetry Collector for centralized data processing
4. Versioning and upgrading OpenTelemetry components:
   • Keep SDKs and collectors up-to-date
   • Use semantic versioning to manage breaking changes

Key Takeaways

• OpenTelemetry APM standardizes telemetry data collection across your entire stack
• It offers vendor-neutral, portable observability that prevents lock-in
• Implementing OpenTelemetry APM improves visibility in complex, distributed systems
• Native OpenTelemetry tools like SigNoz enhance the APM experience with tailored features

FAQs

What are the main differences between OpenTelemetry and traditional APM tools?

OpenTelemetry provides a standardized, vendor-neutral approach to instrumentation and data collection. Traditional APM tools often use proprietary agents and data formats, leading to vendor lock-in. OpenTelemetry allows you to switch between different APM backends without changing your instrumentation code.

How does OpenTelemetry APM improve application performance?

OpenTelemetry APM doesn't directly improve performance, but it provides the detailed insights needed to identify and resolve performance issues. By offering comprehensive tracing, metrics, and logging capabilities, it enables developers to pinpoint bottlenecks, optimize resource usage, and enhance overall application efficiency.

Can OpenTelemetry be used with existing monitoring solutions?

Yes, many existing monitoring solutions have added support for OpenTelemetry data. You can often use OpenTelemetry alongside your current tools, gradually transitioning as you become more comfortable with the new approach. Some providers offer OpenTelemetry exporters or collectors to facilitate this integration.

What skills are needed to implement OpenTelemetry APM effectively?

To implement OpenTelemetry APM effectively, you should have:

1. Familiarity with distributed systems concepts
2. Knowledge of your application's programming language(s)
3. Understanding of observability principles (traces, metrics, logs)
4. Basic DevOps skills for configuring collectors and exporters
5. Data analysis capabilities to interpret and act on the collected telemetry

As you gain experience with OpenTelemetry, you'll develop a deeper understanding of its capabilities and best practices for application monitoring.

Further Reading

SigNoz - an open-source alternative to DataDog

OpenTelemetry Collector - a complete guide