Send Metrics to SigNoz (Self Hosted)

By default, when you install SigNoz, only the Hostmetric receiver is enabled. Before you can query other metrics, you must first enable additional receivers in SigNoz.

There are two ways in which you can send metrics to SigNoz using OpenTelemetry:

• Enable a Specific Metric Receiver
• Enable a Prometheus Receiver
• Find Metrics available in SigNoz
  • Metrics from Hostmetrics receiver
• Related Videos
• Get Help

Depending on your choice, use one of the sections below.

Enable a Specific Metric Receiver

SigNoz supports all the receivers that are listed in the opentelemetry-collector-contrib GitHub repository. To configure a new metric receiver, you must edit the receivers section of the deploy/docker/clickhouse-setup/otel-collector-config.yaml file. The following example shows the default configuration in which the hostmetrics receiver is enabled:

receivers:
  otlp/spanmetrics:
    protocols:
      grpc:
        endpoint: "localhost:12345"
  otlp:
    protocols:
      grpc:
      http:
  jaeger:
    protocols:
      grpc:
      thrift_http:
  hostmetrics:
    collection_interval: 30s
    scrapers:
      cpu:
      load:
      memory:
      disk:
      filesystem:
      network:
processors:
  batch:
    send_batch_size: 1000
    timeout: 10s
# This file was truncated for brevity

To enable a new OpenTelemetry receiver, follow the steps below:

1. Move into the directory in which you installed SigNoz, and open the deploy/docker/clickhouse-setup/otel-collector-config.yaml file in a plain-text editor.
2. Configure your receivers. The following example shows how you can enable a receiver named examplereceiver:

receivers:
  otlp/spanmetrics:
    protocols:
      grpc:
        endpoint: "localhost:12345"
  otlp:
    protocols:
      grpc:
      http:
  jaeger:
    protocols:
      grpc:
      thrift_http:
  hostmetrics:
    collection_interval: 30s
    scrapers:
      cpu:
      load:
      memory:
      disk:
      filesystem:
      network:
  examplereceiver:
    endpoint: 1.2.3.4:8080
processors:
  batch:
    send_batch_size: 1000
    timeout: 10s
# This file was truncated for brevity.

For details about configuring OpenTelemetry receivers, see the README page of the opentelemetry-collector GitHub repository. 3.

Save the changes you’ve made and restart SigNoz. Depending on how you installed SigNoz, follow the steps in one of the sections below:

• Docker Standalone
• Docker Swarm
• Kubernetes

Enable a Prometheus Receiver

SigNoz supports all the exporters that are listed on the Exporters and Integrations page of the Prometheus documentation. If you have a running Prometheus instance, and you expose metrics in Prometheus, then you can scrape them in SigNoz by configuring Prometheus receivers in the receivers.prometheus.config.scrape_configs section of the deploy/docker/clickhouse-setup/otel-collector-config.yaml file.

To enable a Prometheus receiver, follow the steps below:

1. Open the deploy/docker/clickhouse-setup/otel-collector-config.yaml file in a plain-text editor.

2. Enable a new Prometheus receiver. Depending on your use case, there are two ways in which you can enable a new Prometheus exporter:

   • By creating a new job: The following example shows how you can enable a Prometheus receiver by creating a new job named my-new-job:

     receivers:
       otlp:
         protocols:
           grpc:
           http:
     
       # Data sources: metrics
       prometheus:
         config:
           scrape_configs:
             - job_name: "otel-collector"
               scrape_interval: 30s
               static_configs:
                 - targets: ["otel-collector:8889"]
             - job_name: "my-new-job"
               scrape_interval: 30s
               static_configs:
                 - targets: ["localhost:8080"]          
     processors:
       batch:
         send_batch_size: 1000
         timeout: 10s
     # This file was truncated for brevity.
     

   • By adding a new target to an existing job: The following example shows the default otel-collector job to which a new target (localhost:8080) was added:

     receivers:
       otlp:
         protocols:
           grpc:
           http:
     
       # Data sources: metrics
       prometheus:
         config:
           scrape_configs:
             - job_name: "otel-collector"
               scrape_interval: 30s
               static_configs:
                 - targets: ["otel-collector:8889", "localhost:8080"]       
     processors:
       batch:
         send_batch_size: 1000
         timeout: 10s
     # This file was truncated for brevity.
     

   ✅ Info

   To correctly scrape from prometheus instance running on the host machine, replace localhost:8080 with 172.17.0.1:8080 for Linux and host.docker.internal:8080 for MacOS.

   Note that all the jobs are scraped in parallel, and all targets inside a job are scraped serially. For more details about configuring jobs and targets, see the following sections of the Prometheus documentation:

   • <scrape_config>
   • Jobs and Instances

3. Save the changes you’ve made and restart SigNoz. Depending on how you installed SigNoz, follow the steps in one of the sections below:

   • Docker Standalone
   • Docker Swarm
   • Kubernetes

Find Metrics available in SigNoz

You can connect to SigNoz's ClickHouse instance and find the metrics SigNoz is storing. As of now this is bit of manual process, we are bringing capabilities to auto-suggest available metrics soon.

You can follow the below steps:

1. Install ClickHouse client

2. Connect to the ClickHouse container

docker exec -it signoz-clickhouse bash

3. Run the clickhouse-client command to connect to the database service

clickhouse client --host <SigNoz IP>  --port 9000

4. Run the query to list metrics

select DISTINCT(JSONExtractString(time_series_v2.labels,'__name__')) as metrics from signoz_metrics.distributed_time_series_v2

5. If needed, dump in a csv file and parse it locally

select DISTINCT(labels) from signoz_metrics.distributed_time_series_v2 INTO OUTFILE 'output.csv'

You can use this metrics to plot in the Dashboard section.

Metrics from Hostmetrics receiver

Metrics which are available if hostmetrics is enabled. This is enabled in SigNoz default installation.

Related Videos

• How to view Prometheus Metrics in SigNoz

Get Help

If you need help with the steps in this topic, please reach out to us on SigNoz Community Slack.

If you are a SigNoz Cloud user, please use in product chat support located at the bottom right corner of your SigNoz instance or contact us at cloud-support@signoz.io.