-
Notifications
You must be signed in to change notification settings - Fork 183
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Extend GenAI System to include OpenAI compatible platforms #1655
base: main
Are you sure you want to change the base?
Extend GenAI System to include OpenAI compatible platforms #1655
Conversation
6a0bc8d
to
1d76c72
Compare
PS I used the Azure AI Inference client client per instructions to access Azure OpenAI service, and I guess what's happening here more generally, is that Specifically, the Azure AI Inference client can access three different products:
In each case the services are different (different subdomains, for example). However, status quo, all would set So, status quo |
This is definitely an issue that we have faced as well. Right now auto instrumentation default sets genai.system to openai even if the base_url used is different from the openai one. More broadly it's only looking at the client library being used as opposed to the inference server being used, which is the correct representation. In Langtrace's SDK, we capture the underlying server being used from the base url using regex and set a langtrace attribute called
|
Base URL regex is not reliable either - you can run smthing behind the proxy, self-host model in vllm, your provider can host different models behind one API. Instrumentation can do the best effort, but cannot guarantee anything beyond "a system this library was designed for" on the |
Signed-off-by: Adrian Cole <[email protected]>
Signed-off-by: Adrian Cole <[email protected]>
78a512b
to
9300a6d
Compare
@lmolkova want me to fix the "dead link" on https://blogs.oracle.com/linux/post/understanding-linux-kernel-memory-statistics? it is result code zero which I've seen elsewhere when a link takes too long to load or otherwise |
The broken links might not be a new problem. We can work around this by retrying externally, in the make file, but I imagine for now, folks can just manually merge when MLC is flakey on a good link. #332 (comment) |
It has been here since a few and has not been a problem to have things merged :) |
This PR was marked stale due to lack of activity. It will be closed in 7 days. |
This extends GenAI System to include OpenAI compatible platforms. These are usable by OpenAI client libraries:
At the moment, there is no way to attribute signals to Azure OpenAI, so people use "openai". At Elastic, we have different
dashboards for OpenAI (the platform) vs Azure OpenAI, as these are independent services. Lacking a GenAI system name, we have to resort to "openai" which is ambiguous between these two.
Using "server.address" isn't a great option due to subdomains used in the Azure OpenAI service depending on the application name in Azure. While we could use cloud semantics, they aren't guaranteed to be present, and aren't integrated in current code. Even if it were, this it is less ideal to navigate or aggregate on two attributes vs one.
This also adds a value for Google Gemini which has exactly the same concern as it can be accessed either via its native API or via OpenAI libraries.
Changes
Added the following, clarifying only when needed:
Merge requirement checklist
[chore]