-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DRAFT] RTKQ Infinite Query integration #4738
base: master
Are you sure you want to change the base?
Conversation
Review or Edit in CodeSandboxOpen the branch in Web Editor • VS Code • Insiders |
This pull request is automatically built and testable in CodeSandbox. To see build info of the built libraries, click here or the icon next to each commit SHA. Latest deployment of this branch, based on commit db14ff5:
|
size-limit report 📦
|
✅ Deploy Preview for redux-starter-kit-docs ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
@remus-selea you reported over in #4393 (comment) that:
I just tried reproducing that and I'm not seeing it happen with the current PR. It's entirely possible my test doesn't match what you were doing. Could you give me a repro? |
Knocked out |
Added an example app that ports over the main React Query sandboxes for infinite queries. (Technically "pagination" didn't use an infinite query, and in fact Dominik says you shouldn't use an infinite query for pagination, but I'd already gotten something working so I kept it.) In the process I found out that:
|
As I worked on creating a demo of the bug I discovered more details. The bug seems to only happen for queries that take no arguments. Codesandbox where the issue occurs: https://codesandbox.io/p/sandbox/spring-star-dtc2gy |
hi, I have been using rtk-query, but now I need this function urgently. Is there a timeline for releasing the beta version for infinite query? |
@guaizi149 : no timeline, but the code in this PR works enough that we're asking people to try it out, let us know if they see bugs, and offer feedback on the API design. See the top comment for installation instructions (and be sure to actually look in the CodeSandbox CI job for the actual latest commit installation command). |
Is there any way to access the input parameter provided to the infinite query hook inside of the endpoint definition's Test.tsx
query.ts
|
@agusterodin : you'd have to change the page param type to be |
Oops, my example was screwed up. I meant to have the endpoint generics set up like this:
Are you suggesting to do this? I must misunderstand how infinite queries work because this feels like a workaround.
|
@agusterodin : I think that looks right, yeah. Think of it this way: One other nuance here: There ends up being a separation between the "cache key" value, which is still what you pass to the query hook, and the "initial page param" value. In your case, I think they end up being related - ie, you probably do want to use the filters as the cache key ("all the pages are results for this set of filters"), and then also pass them as the override initial hook value: // The overall cache entry containing _all_ the pages
// will be keyed by serializing `filters`
const { data } = useMyInfiniteQuery(filters, {
infiniteQueryOptions: {
// in order to fetch any given page, we need both `filters` and `offset`
initialPageParam: {filters, offset: 0}
}
}) But yeah, questions like this are also why we're asking people to try this out :) |
Attempting to "port" this Tanstack Query/Virtual infinite scroll example to use RTKQ. Assuming this would be a relatively common use-case. I got stuck at Not sure if it is helpful, but leaving this here to show how i'm attempting to use the infinite query implementation. Will continue testing against future versions and leave feedback! InfiniteQueryTest.tsx
|
@agusterodin and yes, we don't have |
0f1db3d
to
a28de8f
Compare
@agusterodin I just put up a PR implementing all of the infinite query status flags over in #4771 went ahead and merged in the PR - it also fixed a types error after I rebased this branch, and I feel good about the implementation. |
Just tried commit 33e30af. Sorry for delayed response. For some reason the network requests being made to get the "next page" are always the same as the initial request (first page, offset of 0). Note that the code below is based off of the example you provided under the examples directory (not using Tanstack Virtual). InfiniteQueryTest.tsx (barely modified from original example). The only noteworthy difference is that I provide
infiniteQueryApi.ts. Our backend doesn't use cursors, we have an offset (starting index of items to return) and limit (amount of items to return).
Notice from the below screenshot that the Also, another general API question: I have two PS: sorry that all I have is a shitty screenshot and code snippets. I was contemplating setting up a minimal Vite project, but I wouldn't be able to have it send requests to our backend because it is hitting an authenticated endpoint which makes the setup hard to recreate. I would have attached a video screenshot at the bare minimum, but the MacOS video screenshot tool is apparently broken in the current version of MacOS. If what I provided doesn't just contain a trivial error and isn't adequate to help you identify the issue, I will set something better up. |
@agusterodin yeah, if you could give me a project that shows the inconsistent next page behavior, that would definitely help!
As with the other hook options, providing an option in the hook overrides the default value supplied in the endpoint. Think of it as the general equivalent of a default value for a function argument or destructured object field - if you don't pass it in the hook, we have to have something to fall back to. |
Identified issue. If you have the global Uncomment line https://github.com/agusterodin/rtkq-infinite-query-playground |
…uerying will happen
…of args and forward/backwards direction
…Page functions + Thunk
# Conflicts: # packages/toolkit/src/query/core/buildThunks.ts # packages/toolkit/src/query/index.ts # packages/toolkit/src/query/react/buildHooks.ts
* Add blank RTK app * Nuke initial examples * Add react-router * Basic routing * Add msw * Configure MSW * Set up MSW * Use RTK CSB CI build * Add baseApi * Add pagination example * Add react-intersection-observer * Add infinite scroll example * Add max-pages example * Drop local example lockfile * Drop back to Vite 5 to fix TS issue * Align Vite versions to fix test TS error
* Extract InfiniteQueryDirection * Export page param functions * Fix useRefs with React 19 * Fix infinite query selector arg type * Implement infinite query status flags * Fix types and flags for infinite query hooks * Add new error messages
87270fe
to
79f6ff8
Compare
…4795) * Consolidate test assertions * Add failing tests for infinite queries vs refetching * Tweak infinite query forced check
@agusterodin Thanks for the repro, that made it pretty easy to identify the cause. The internal logic had this sequence: // Start by looking up the existing InfiniteData value from state,
// falling back to an empty value if it doesn't exist yet
const blankData = { pages: [], pageParams: [] }
const cachedData = getState()[reducerPath].queries[arg.queryCacheKey]
?.data as InfiniteData<unknown, unknown> | undefined
const existingData = (
isForcedQuery(arg, getState()) || !cachedData ? blankData : cachedData
) as InfiniteData<unknown, unknown> Problem is that function isForcedQuery(
arg: QueryThunkArg,
state: RootState<any, string, ReducerPath>,
) {
const requestState = state[reducerPath]?.queries?.[arg.queryCacheKey]
const baseFetchOnMountOrArgChange =
state[reducerPath]?.config.refetchOnMountOrArgChange
const fulfilledVal = requestState?.fulfilledTimeStamp
const refetchVal =
arg.forceRefetch ?? (arg.subscribe && baseFetchOnMountOrArgChange)
if (refetchVal) {
// Return if it's true or compare the dates because it must be a number
return (
refetchVal === true ||
(Number(new Date()) - Number(fulfilledVal)) / 1000 >= refetchVal
)
}
return false
} So, it was always opting to use I spent a while staring at it, and eventually changed the line to use a different flag we set internally when you actually call (also I figured out that we have two different |
* Add bidirectional cursor infinite scroll example * Add example using limit and offset - Add example using page and size - add an intersection observer callback ref hook * Bump infinite query RTK version --------- Co-authored-by: Mark Erikson <[email protected]>
@remus-selea added several more examples:
|
* Fix updateQueryData for infinite queries * Fix upsertQueryData for infinite queries * Fix upsertQueryEntries for infinite queries * Fix types in lifecycle middleware
…le (#4798) * Bump infinite query RTKQ build * Add rn-web and force infinite query example lockfile * Bump MSW worker * Add rn-web types * Expose refetch from infinite query hooks * Add RN FlatList example * Update root lockfile
Pretty significant progress today!
Also did some checks on bundle size. Looks like the new functionality adds 7K min to all RTKQ usages (but only about 1.5K min+gz). The bad news is that's non-negotiable and the increase applies even if you never use an infinite query endpoint, due to the changes currently being deeply embedded inside all of the RTKQ core logic. That said, this PR started with a lot of intentional copy-paste duplication in order to get things working at all. I think I can manage some deduplication and maybe shave off about 2K min of that. FWIW, it looks like React Query's implementation of infinite queries is about 3K min, so this isn't that far off. Theirs is partially shakeable. My general intent at this point is that we'll end up shipping roughly the architectural approach that's in this PR, minus whatever deduplication and byteshaving I can do as cleanup before I decide it's ready. I'm always sensitive about bundle size increases, so I certainly wish I had an easy way to make the bundle size cost opt-in and only if you actually have some infinite query endpoints. At the moment, and with the way this PR is currently architected, I don't have a good idea for how to do that. Longer-term, I've been vaguely tossing around some ideas for seeing if we can pull out some of our functionality based on RTKQ's existing "modules" concept. Currently we just have the "core" and "React" modules. I don't know if it's feasible to make this more shakeable or opt-in, but it's worth investigating. That would happen as part of a future RTK 3.0 effort, no ETA. All that said, given the amount of actual functionality involved here, 5K min seems like a plausibly reasonable cost to pay to add the feature. |
Overview
This is a running integration PR for the upcoming RTK Query support for "infinite / paginated queries".
The overall plan is to reimplement React Query's public API for infinite queries on top of RTK Query's internals.
This work was started by @riqts in [API Concept] - Infinite Query API. I picked up that work, rebased it, and have fleshed out the TS types, tests, and implementation. See that PR for the original discussion and progress. Also, thanks to @TkDodo for the advice on how they designed the infinite query API for React Query!
My current somewhat ambitious goal is to ship a final version of infinite query support for RTKQ by the end of this year. I am absolutely not going to guarantee that :) It's entirely dependent on how much free time I have to dedicate to this effort, how complicated this turns out to be, and how much polish is needed. But in terms of maintenance effort, shipping this is now my main priority!
Status
It currently works sufficiently that it's ready to be tried out in example apps so that we can confirm it works across a variety of use cases, and find bugs or areas that need work. It's not ready for an official alpha yet.
You can try installing the PR preview build using the installation instructions from the "CodeSandbox CI" job listed at the bottom of the PR. Please leave comments and feedback here!
I've got a laundry list of examples and tests that ought to be added - see the "Todos" section below. Contributions are appreciated!
Preview Builds
Open the "CodeSandbox CI" details check:
Click the most recent commit on the left, then copy-paste the install instructions for your package manager from the "Local Install Instructions" section on the right, such as:
Todos
Jotting down some todos as I think of them:
Functionality
gN/PPP
whenmaxPages
> 0isFetchingNext/PreviousPage
flagshasNext/PreviousPage
flagsInvestigate movingThis probably still makes sense to keep with the data in case of users upsertingpageParams
into some new metadata field in the cache entry, so that it's not directly exposed to the user indata
(per discussion with Dominik)combinePages
option, so that you don't have to doselectFromResult: ({data}) => data.pages.flat()
(and memoize it) for every endpointcreateSelector
by default?Tests / Example Use Cases
React Query examples
ref:
Other
upsertQueryData
/upsertQueryEntries
?