[BREAKING] New Search API Result Limits

Have more questions? Submit a request

13 Comments

  • Anton Mintsev
    Comment actions Permalink

    Hi!, will updated search endpoint still return the total number of search results when it's more than 1000?

    0
  • Ali Ayaz
    Comment actions Permalink

    Same question as the one Anton asked. And related to it...

    Does this mean that there will be no way for us to grab results using the search api end point, when the results are greater than 1000?? No pagination, nothing?

    I hope this is reconsidered and/or a viable alternative is offered — I for one use this end point quite often (e.g. we have close to 300k users, and if we want to just grab a subset of those users, it’s easier to do so via the search api ep — however those users will be more than 10k at times and it’s almost impossible to further bifurcate them ... and using the users api ep in this case is extremely inefficient)

    4
  • Salvador Vazquez
    Comment actions Permalink

    @Anton: You will still be returned the count of all items matching the search with your request as you had before

    @Ali: With this change you will not be able to get any results beyond 1k. The idea of the search API is to get the top results of the search instead of all items. Can you give me a bit more information about your use case of getting that list of users and why the user API would be inefficient for you. 

    0
  • Daniel Ertman
    Comment actions Permalink

    @Salvador : Not to speak for Ali, but here's our situation : Sometimes, something in our system goes down. When that happens, Care can get flooded with Zendesk tickets - we've seen 10,000 - 20,000 tickets during some particularly bad outages. It would be impossible for humans to deal with those tickets one by one, so we just close all of the tickets and deal with the customers / orders en masse.

    To close all of the tickets, we do a search by time range, particular tags, and particular custom fields. There's nothing more specific we can do. We're going to get a result of 10,000 tickets, because there are 10,000 tickets we need to close.

    With this change, it seems like we will need to:

    1. Run the search, get the total number of results
    2. Guess at a time segmentation that will result in less than 1000 tickets
    3. Run many searches to re-accumulate all of those results
    4. If we were wrong and any search returns more than 1000 results, split that time window and try again.

    This isn't better than paging for us. If you don't want us to use the search for this use-case, that's fine, but we'd need something more than time range added to incremental export in order to only close tickets impacted by our outage.

    0
  • Ali Ayaz
    Comment actions Permalink

    @Salvador: grabbing 300k users, in total, iterating through them to only display those records in a g.sheet which meet a certain criterion (having a certain tag, etc) VS using the search api ep, which will only return those users that meet the required criterion and the script won’t be facing timeouts in the latter case, whereas those would be common in the former case. Furthermore, the former case means more api requests, while the later case is faster, and significantly less api requests....

    This is just ONE of the many use cases... believe you me, we have dozens more such examples.

    Additionally, one other point, which isn’t mentioned anywhere... this change will also impact the regular search in the Zendesk Support Product (for users, tickets, organizations, etc) — won’t it?
    I honestly believe that this will be a major pain point, if not for all, but at the very least for most of your customers, especially the ones who are utilising this particular api ep, for tickets, orgs, users, etc. We have several Zd instances in the organization and our main/largest instance has over a million tickets. The search api ep made it easier to export data out based on a certain criteria (a custom field value, tag, group, etc) and now we will have to face major overhaul of our approach towards all our Zd specific scripts and integrations, etc.

    2
  • Alex Pereira
    Comment actions Permalink

    Hi,

    Could someone please clarify whether or not this change will affect any of these end-points as well:

    /api/v2/users/search.json?query={query}

    https://developer.zendesk.com/rest_api/docs/support/users#search-users

    /api/v2/organizations/search.json?external_id={external_id}

    https://developer.zendesk.com/rest_api/docs/support/organizations#search-organizations-by-external-id

     

    Thanks!

    0
  • Ali Ayaz
    Comment actions Permalink

    @Alex — of course it will impact those endpoints. You will only get the first 1000 results returned in a single page, with absolutely no option for pagination. That’s the way this change will be rolled out, as per the standing currently. Here’s to hoping that this will revisited by Zendesk.

    1
  • Graeme Carmichael
    Comment actions Permalink

    Will there be anything in the web response to indicate that results are incomplete? (Other than hitting exactly 1000 items)-sorry. I see you have answered that. 

    0
  • Dustin King
    Comment actions Permalink

    Beyond all of the API uses mentioned above (we have some ourselves), assuming this affects Zendesk Support's own search functionality, this will be a major hindrance to our business. When an issue in one of our products arises, we sometimes receive well over 1K tickets about the issue, and need to be able to search and find them all for triage, bug identification/solving, and most importantly for reply and remediation for our customers. Depending on the issue, we don't always have a specific time frame to search within to try and narrow those results to do this in batches, and the incident might be over a larger time frame.

    Shouldn't there be at least a secondary endpoint that returns a larger number of results?

    0
  • Igal Dar
    Comment actions Permalink

    Lowering the limit from 1000 to 100 per page, and max results is 1000 results (10 pages) is disappointing.
    It means many of the real-time data is maxed at 1000. Explore does not process real time data, leaving search, views, etc.

    I was very excited to read the previous announcement of maximum results per page 1000. It allowed us to simplify a lot of our custom code, eliminating much unneeded pagination. For example - a ticket can have more than 100 comments, but unlikely to have more than 1000 comments. If it has, only the last 1000 are relevant. One query would have gotten all relevant comments. Now we're rethinking pagination.

    It would be great if this can be overturned. Or have a way that curtain customers, with curtain offering would benefit from 1000 results per page and 200K results.

    0
  • Jesse Rubenfeld
    Comment actions Permalink

    I'm confused. People are talking about no pagination, but the notice above references a max results per page of 100 (out of the 1,000 total maximum results). So it looks like we do still need to use pagination, and there's the total results limit. Maybe the change has changed? Can you please clear up the confusion? Did you previously advertise all 1,000 results in a single page and now have changed it to 10 pages of 1,000? (If so, why?) 

    1
  • Alex Pereira
    Comment actions Permalink

    @Jesse Rubenfeld, that seems to be exactly what happened. They have changed the narrative of the original notice. Originally, it was 1,000 results in a single page, no pagination...

    1
  • Jesse Rubenfeld
    Comment actions Permalink

    Will the tenth 100-result page of a search with 3,254 results have a null "next_page" value?

    0

Please sign in to leave a comment.

Powered by Zendesk