Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add_opentelementry_sqlcommenter_package #440

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/hex-packages.json
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,15 @@
"language": "elixir",
"authorizedUsers": ["bryannaegele", "tsloughter"]
},
"sqlcommenter": {
"workingDirectory": "instrumentation/opentelemetry_sqlcommenter",
"name": "SQLCommenter Instrumentation",
"packageName": "opentelemetry_sqlcommenter",
"tagPrefix": "opentelemetry-sqlcommenter-v",
"buildTool": "mix",
"language": "elixir",
"authorizedUsers": ["bryannaegele", "dkuku", "tsloughter"]
},
"tesla": {
"workingDirectory": "instrumentation/opentelemetry_tesla",
"name": "Tesla Instrumentation",
Expand Down
3 changes: 3 additions & 0 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,9 @@ opentelemetry_redix:
opentelemetry_req:
- instrumentation/opentelemetry_req/**/*

opentelemetry_sqlcomenter:
- instrumentation/opentelemetry_sqlcomenter/**/*

opentelemetry_telemetry:
- utilities/opentelemetry_telemetry/**/*

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
_extends: opentelemetry-erlang-contrib:.github/release-drafter.yml
name-template: 'Opentelemetry SQLCommenter - v$RESOLVED_VERSION'
tag-template: 'opentelemetry-sqlcommenter-v$RESOLVED_VERSION'
tag-prefix: opentelemetry-sqlcommenter-v
include-paths:
- instrumentation/opentelemetry_sqlcommenter/
39 changes: 39 additions & 0 deletions .github/workflows/elixir.yml
Original file line number Diff line number Diff line change
Expand Up @@ -490,6 +490,45 @@ jobs:
- name: Test
run: mix test

opentelemetry-sqlcommenter:
needs: [test-matrix]
if: (contains(github.event.pull_request.labels.*.name, 'elixir') && contains(github.event.pull_request.labels.*.name, 'opentelemetry_req'))
env:
app: "opentelemetry_sqlcommenter"
defaults:
run:
working-directory: instrumentation/${{ env.app }}
runs-on: ubuntu-24.04
name: Opentelemetry SQLCommenter test on Elixir ${{ matrix.elixir_version }} (OTP ${{ matrix.otp_version }})
strategy:
fail-fast: false
matrix: ${{ fromJson(needs.test-matrix.outputs.matrix) }}
steps:
- uses: actions/checkout@v4
- uses: erlef/setup-beam@v1
with:
version-type: strict
otp-version: ${{ matrix.otp_version }}
elixir-version: ${{ matrix.elixir_version }}
rebar3-version: ${{ matrix.rebar3_version }}
- name: Cache
uses: actions/cache@v4
with:
path: |
~/deps
~/_build
key: ${{ runner.os }}-build-${{ matrix.otp_version }}-${{ matrix.elixir_version }}-v3-${{ hashFiles('**/mix.lock') }}
- name: Fetch deps
if: steps.deps-cache.outputs.cache-hit != 'true'
run: mix deps.get
- name: Compile project
run: mix compile --warnings-as-errors
- name: Check formatting
run: mix format --check-formatted
if: matrix.check_formatted
- name: Test
run: mix test

opentelemetry-telemetry:
needs: [test-matrix]
if: (contains(github.event.pull_request.labels.*.name, 'elixir') && contains(github.event.pull_request.labels.*.name, 'opentelemetry_telemetry'))
Expand Down
1 change: 1 addition & 0 deletions CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
/instrumentation/opentelemetry_oban @indrekj
/instrumentation/opentelemetry_phoenix @bryannaegele @tsloughter
/instrumentation/opentelemetry_redix @andrewhr
/instrumentation/opentelemetry_sqlcommenter @bryannaegele @dkuku @tsloughter
/instrumentation/opentelemetry_xandra @whatyouhide
/utilities/opentelemetry_telemetry @bryannaegele @tsloughter
/utilities/otel_http @bryannaegele @tsloughter
Expand Down
4 changes: 4 additions & 0 deletions instrumentation/opentelemetry_sqlcommenter/.formatter.exs
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Used by "mix format"
[
inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"]
]
119 changes: 119 additions & 0 deletions instrumentation/opentelemetry_sqlcommenter/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# OpentelemetrySqlcommenter

OpentelemetrySqlcommenter is an Elixir library that automatically adds
OpenTelemetry trace context to your SQL queries as comments. This enables
better observability by linking database operations with distributed
traces.

> **Note:** Currently only compatible with PostgreSQL databases.

## Features

- Automatically injects OpenTelemetry trace context into SQL queries
- Compatible with Ecto repositories using PostgreSQL
- Follows W3C Trace Context specification
- Minimal performance overhead
- Easy integration with existing projects

## Installation

Add `opentelemetry_sqlcommenter` to your list of dependencies in `mix.exs`:

```elixir
def deps do
[
{:opentelemetry_sqlcommenter, "~> 0.1.1"},
]
end
```

## Usage

1. In your Ecto repository, delegate the `prepare_query` function to OpentelemetrySqlcommenter:

```elixir
defmodule YourApp.Repo do
use Ecto.Repo,
otp_app: :your_app,
adapter: Ecto.Adapters.Postgres

defdelegate prepare_query(operation, query, opts), to: OpentelemetrySqlcommenter
end
```

2. That's it! Your SQL queries will now include trace context as comments:

```
2024-12-16 18:25:32.115
GMT,"postgres","ai_connector_dev",104871,"127.0.0.1:58162",67607098.199a7,1,
"SELECT",2024-12-16 18:25:28 GMT,9/78,0,LOG,00000, "execute
<unnamed>: SELECT u1.""id"", u1.""email"", FROM ""user_token""
AS u0 INNER JOIN ""user"" AS u1 ON u1.""id"" = u0.""user_id""
WHERE ((u0.""token"" = $1) AND (u0.""context"" = $2)) AND
(u0.""inserted_at"" > $3::timestamp + (-(60)::numeric * interval '1
day'))/*traceparent='00-6b6bcfabaea5ef9890481e9b0c18efac-4deda0f3c249b6c7-01'*/",
"parameters: $1 =
'\x020aec1aef94f66f64dd5213ed23e4b244cf40f4dae502ebae1d47857982b99b',
$2 = 'session', $3 = '2024-12-16 18:25:32.090091'" ,,,,,,,,"","client
backend",,0
```

## Configuration

No additional configuration is needed if you already have OpenTelemetry set up in your application.

## How It Works

The library:
1. Intercepts queries before they're sent to the database
2. Checks for active OpenTelemetry trace context
3. If present, adds the trace context as a SQL comment
4. Preserves all existing query options

## Important Disclaimer

⚠️ **Impact on Query Performance**

This library disables prepared statements and query caching by setting
`prepare: :unnamed` for all queries. This is necessary because:

1. SQL comments make each query unique, even if the underlying SQL is identical
2. Prepared statements and caching rely on query text matching exactly
3. Adding trace context makes each query instance unique,
defeating the purpose of preparation and caching

### Performance Implications

- **Prepared Statements**: Normally, Postgres can prepare and cache
query execution plans. With this library, each query is treated as new,
requiring a new execution plan.

### When to Use

Use this library when:
- Trace context in SQL logs is more important than maximum query performance
- You're debugging or need detailed observability
- Your application's performance isn't critically dependent on prepared statements

Consider disabling it in performance-critical environments or creating a configuration option to toggle it based on your needs.

### Alternative Approaches

If you need both tracing and prepared statements, consider:
1. Using separate logging mechanisms for trace context
2. Implementing selective tracing for specific queries only
3. Using database-specific audit logging features

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## Related Links

- [SQL Commenter with Postgrex](https://dev.to/dkuku/sql-commenter-with-postgrex-2bfd)
- [OpenTelemetry Documentation](https://opentelemetry.io/docs/)
- [W3C Trace Context Specification](https://www.w3.org/TR/trace-context/)

## Documentation

Full documentation can be found at [HexDocs](https://hexdocs.pm/opentelemetry_sqlcommenter).
Loading
Loading