Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split chunks if their queries time out #12

Open
vcschapp opened this issue Dec 14, 2021 · 0 comments
Open

Split chunks if their queries time out #12

vcschapp opened this issue Dec 14, 2021 · 0 comments
Assignees
Labels
easy Not a hard problem to solve enhancement New feature or request

Comments

@vcschapp
Copy link
Collaborator

vcschapp commented Dec 14, 2021

User Story

As an application developer, I want increased confidence that I'm getting all available results from Insights even when my queries process so much data that they time out.

Notes

  1. This feature would build on the existing splitting added in issue Split chunks when max result limit is reached #3. Instead of only being able to split when we get MaxLimit results, we also split when we get a timeout error back from CWL Insights on a GetQueryResults poll.
  2. Part of this feature should include allowing arbitrary client-side timeouts for chunks. The current CWL timeout is 15m, which is a very long time to run to get feedback of this nature. When running a Query, customers may want to set lower timeouts so they can split faster and have some hope of quickly whittling oversize time ranges to achievable levels.

Open Questions

  1. Add a new field into QuerySpec to request this, or just do it automatically when splitting due to result count is enabled?
@vcschapp vcschapp changed the title Split chunks when chunk times out Split chunks if their queries time out Dec 14, 2021
@vcschapp vcschapp added enhancement New feature or request easy Not a hard problem to solve labels Feb 8, 2022
@vcschapp vcschapp self-assigned this Feb 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
easy Not a hard problem to solve enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant