GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Improved batch processing by UmanShahzad · Pull Request #37 · ipinfo/python

Conversation

@UmanShahzad

@UmanShahzad UmanShahzad commented Dec 16, 2020

Copy link
Copy Markdown
Contributor

Closes #33
Closes #35

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@UmanShahzad
refactor common handler utilities out
3b1bbda
@UmanShahzad UmanShahzad self-assigned this Dec 16, 2020
UmanShahzad added 5 commits December 16, 2020 11:12
@UmanShahzad
comments for util funcs
4e346ea
@UmanShahzad
note most recent changes to changelog
c9cabb1
@UmanShahzad
update requirements
74f7ec5
@UmanShahzad
put SDK version in a common place
7cfafa9
@UmanShahzad
address #33
5d363ea
also get rid of useless tests that only mock stuff.
@UmanShahzad UmanShahzad mentioned this pull request Dec 16, 2020
Create iterator for batch operation #38
Closed
@UmanShahzad

UmanShahzad commented Dec 16, 2020

Copy link
Copy Markdown
Contributor Author

Basic improvements to the batch processor work now, specifically that chunking happens and so there is no real limit to the input list size.

What's next:

  • Add an option which if turned on, a 429 status on one of the responses doesn't discard all successful responses so far.
  • Add a scaling and hard timeout, and options to override them.

From this I realized we can create an iterator-based batch function, so filed #38. The existing non-iterators can be refactored to reuse the iterator later by consuming all of the iterator at once before returning.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

UmanShahzad added 8 commits December 16, 2020 14:21
@UmanShahzad
basic batch improvements.
14d1ceb
chunking happens now; input list size limit doesnt exist.
@UmanShahzad
support scaling/hard timeouts & graceful-fail for sync batch
901fcaf
@UmanShahzad
prepare changelog for release after async is ready
91fc4a6
@UmanShahzad
clarify docs and changelog
62fe3ed
@UmanShahzad
fixes & impl for async
d815106
@UmanShahzad
fix docs, add support for sync timeout override
c3e5855
@UmanShahzad
test cases and more robust err handling
4347bfd
@UmanShahzad
update readme with input size info
6f36516
@UmanShahzad UmanShahzad changed the title [WIP] Improved batch processing Improved batch processing Dec 21, 2020
@UmanShahzad
UmanShahzad merged commit e229f52 into master Dec 21, 2020
@UmanShahzad
UmanShahzad deleted the uman/better-batch-ops branch December 21, 2020 16:17
a7mad-0
a7mad-0 reviewed Jan 22, 2021
View reviewed changes

@a7mad-0 a7mad-0 left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahmad//aa

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Raldyn15
Raldyn15 approved these changes Oct 12, 2024
View reviewed changes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers
@a7mad-0 a7mad-0 a7mad-0 left review comments
@Raldyn15 Raldyn15 Raldyn15 approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@UmanShahzad UmanShahzad

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Improve batch processing functionality and timeouts handling with large amounts of data Make synchronous code format details first and then cache

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@UmanShahzad @a7mad-0 @Raldyn15
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.