GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Fix wrong buffer size calculation. by buptliuhs · Pull Request #494 · microsoftgraph/msgraph-sdk-javascript

Conversation

@buptliuhs

@buptliuhs Tony Liu (buptliuhs) commented Aug 6, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Summary

The calculation of Buffer size is incorrect (in both v2 and v3).

Motivation

When using OneDriveLargeFileUploadTask to upload small files (usually < 8KB, Buffer.poolSize), buffer size and slice are not calculated correctly, which causes upload failure.

Example of the issue

When uploading a tiny jpeg file whose size is 747 bytes.
Buffer passed in OneDriveLargeFileUploadTask is:
{ byteOffset: 5808, byteLength: 747, length: 747, poolSize: 8192 }

In this case, byteOffset is 5808 and byteLength is 747.
The old code calculates size = byteLength - byteOffset, which is obviously incorrect.

Because of this, the HTTP header that is used later to upload slices will be like:

{
    "content-length": "0",
    "content-range": "bytes 0--5062/-5061",
    ....
}

And it causes a 400 response.

It's likely to happen when the buffer size is less than the pollSize. When buffer is large, byteLength is usually 0.

Test plan

Upload small files (< 8KB) using OneDriveLargeFileUploadTask.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • I have read the CONTRIBUTING document.
  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@nikithauc
Nikitha Chettiar (nikithauc) self-requested a review August 6, 2021 05:17
@nikithauc

Nikitha Chettiar (nikithauc) commented Aug 6, 2021

Copy link
Copy Markdown
Contributor

Tony Liu (@buptliuhs) Thank you for the contribution!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nikithauc

Nikitha Chettiar (nikithauc) commented Aug 6, 2021

Copy link
Copy Markdown
Contributor

Tony Liu (@buptliuhs)

  1. Can you please provide the error and the steps to reproduce the error?
  2. Would it possible to write a unit test identifying this change?
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@buptliuhs

Tony Liu (buptliuhs) commented Aug 6, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

hi Nikitha Chettiar (@nikithauc),

What I did to reproduce the issue is that uploading a tiny jpeg file whose size is 747 bytes.
Buffer passed in OneDriveLargeFileUploadTask is:
{ byteOffset: 5808, byteLength: 747, length: 747, poolSize: 8192 }

In this case, byteOffset is 5808 and byteLength is 747.
The old code calculate size = byteLength - byteOffset, which is obviously incorrect.

Because of this, the http header will be like:

{
    "content-length": "0",
    "content-range": "bytes 0--5062/-5061",
    ....
}

It's likely to happen when the buffer size is less than the pollSize. When buffer is large, byteLength is usually 0.

I'll see how to add a unit test to cover this change.

Cheers,
Tony

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ghost

Deleted user (ghost) commented Aug 7, 2021 •
edited by ghost
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

CLA assistant check
All CLA requirements met.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@buptliuhs

Tony Liu (buptliuhs) commented Aug 7, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Nikitha Chettiar (@nikithauc) I had to move some code out of the create method to make the change testable. Hope it is okay.

nikithauc reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@buptliuhs

Tony Liu (buptliuhs) commented Aug 10, 2021

Copy link
Copy Markdown
Contributor Author

hi Nikitha Chettiar (@nikithauc) , are you happy with the change?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

nikithauc
Nikitha Chettiar (nikithauc) previously approved these changes Aug 10, 2021
View reviewed changes
@nikithauc
Nikitha Chettiar (nikithauc) changed the base branch from v2/dev to dev August 10, 2021 22:54
@nikithauc
Nikitha Chettiar (nikithauc) dismissed their stale review August 10, 2021 22:54

The base branch was changed.

@nikithauc

Nikitha Chettiar (nikithauc) commented Aug 10, 2021

Copy link
Copy Markdown
Contributor

Tony Liu (@buptliuhs) the changes look good. I changed the base to dev for this PR.

Can you please resolve this?

I should have noticed the base setting before. I apologize for the inconvenience.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

nikithauc
Nikitha Chettiar (nikithauc) previously approved these changes Aug 10, 2021
View reviewed changes
@buptliuhs

Tony Liu (buptliuhs) commented Aug 11, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Hi Nikitha Chettiar (@nikithauc), I was initially looking for a fix in v2, which branch should the change be based on? I can create another PR to propagate the fix to dev too to fix the same issue in v3.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@buptliuhs
Tony Liu (buptliuhs) requested a review from Nikitha Chettiar (nikithauc) August 11, 2021 11:35
@nikithauc

Nikitha Chettiar (nikithauc) commented Aug 11, 2021

Copy link
Copy Markdown
Contributor

Tony Liu (@buptliuhs) Can you please branch off dev? The fix will be reflected in 3.x versions.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@buptliuhs
Tony Liu (buptliuhs) dismissed Nikitha Chettiar (nikithauc)’s stale review via ca737ac August 11, 2021 20:42
@buptliuhs
Tony Liu (buptliuhs) force-pushed the v2-fix-wrong-buffer-size-calculation branch from 6fca47e to ca737ac Compare August 11, 2021 20:42
@buptliuhs

Tony Liu (buptliuhs) commented Aug 11, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

Hi Nikitha Chettiar (@nikithauc) , I've done the rebase on the branch and now the branch is based on dev. Please review.
There are some auto-changes as following, hope they are okay:

  • package-lock.json: auto-updated when running npm install
  • samples/*: auto-updated when running git commit

Again, I'm also looking for the same fix on 2.x since 3.x is still in the preview stage. Is 2.x still in maintenance mode? If yes, which branch should I target? Thanks!

Cheers,
Tony

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nikithauc

Nikitha Chettiar (nikithauc) commented Aug 11, 2021

Copy link
Copy Markdown
Contributor

Tony Liu (@buptliuhs) 3.0.0 is released - https://www.npmjs.com/package/@microsoft/microsoft-graph-client/v/3.0.0. 2.x version will no longer be seeing any changes.

buptliuhs reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

nikithauc
Nikitha Chettiar (nikithauc) reviewed Aug 11, 2021
View reviewed changes
Comment thread .gitignore Resolved

Uh oh!

There was an error while loading. Please reload this page.

@buptliuhs
Tony Liu (buptliuhs) force-pushed the v2-fix-wrong-buffer-size-calculation branch from ca737ac to bf5ef40 Compare August 12, 2021 02:16
@buptliuhs
Fix wrong buffer size calculation.
5487143
@buptliuhs
Tony Liu (buptliuhs) force-pushed the v2-fix-wrong-buffer-size-calculation branch from 492b0a7 to 5487143 Compare August 12, 2021 02:25
@buptliuhs

Tony Liu (buptliuhs) commented Aug 12, 2021

Copy link
Copy Markdown
Contributor Author

Just did a final tidy-up.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

nikithauc
Nikitha Chettiar (nikithauc) approved these changes Aug 12, 2021
View reviewed changes
@nikithauc

Nikitha Chettiar (nikithauc) commented Aug 12, 2021

Copy link
Copy Markdown
Contributor

Tony Liu (@buptliuhs) Thank you for this contribution! Merging this right now :)

buptliuhs reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nikithauc
Nikitha Chettiar (nikithauc) merged commit e17274b into microsoftgraph:dev Aug 12, 2021
@buptliuhs
Tony Liu (buptliuhs) deleted the v2-fix-wrong-buffer-size-calculation branch August 12, 2021 02:52
@buptliuhs

Tony Liu (buptliuhs) commented Oct 27, 2021

Copy link
Copy Markdown
Contributor Author

Nikitha Chettiar (@nikithauc) hello again! Do we have a plan for a new release, e.g. 3.0.1, to include the fixes for 3.0.0? Thanks!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nikithauc

Nikitha Chettiar (nikithauc) commented Oct 27, 2021

Copy link
Copy Markdown
Contributor

Tony Liu (@buptliuhs) yes. Please expect 3.0.1 soon

buptliuhs reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@nikithauc Nikitha Chettiar (nikithauc) mentioned this pull request Nov 15, 2021
Release 3.0.1 #532
Merged
@nikithauc

Nikitha Chettiar (nikithauc) commented Nov 15, 2021

Copy link
Copy Markdown
Contributor

Tony Liu (@buptliuhs) https://www.npmjs.com/package/@microsoft/microsoft-graph-client/v/3.0.1

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@nikithauc Nikitha Chettiar (nikithauc) nikithauc approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@buptliuhs @nikithauc
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.