GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Fix `Move-Item` for FileSystemProvider to use copy-delete instead of move for DFS paths by SteveL-MSFT · Pull Request #14913 · PowerShell/PowerShell

Conversation

@SteveL-MSFT

@SteveL-MSFT Steve Lee (SteveL-MSFT) commented Feb 27, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

PR Summary

When using Move-Item between DFS paths, the FileSystemProvider will decide whether to use the .NET Move() or a Copy() and Delete() operation based on comparing the root path of the source and destination. This will work whenever the source and destination are on the same volume, but results in an Access Denied error message if not. In the case of DFS paths, they look like SMB share paths but typically are on different volumes so the move fails with Access Denied.

There is a Win32 API that could be used to determine if a UNC path is DFS, but it appears there is a timeout of around 1 second when the path is NOT DFS, so that's too big of an impact for normal usage.

Fix is to try MoveTo() as that is much faster on the same volume. If that fails, we check the HResult to see if it's Access Denied (0x80070005) and if so (as this is what is returned in DFS scenario) we fallback to CopyAndDelete().

Tested manually on a VM with DFS shares.

PR Context

Reported through Microsoft customer support

PR Checklist

  • PR has a meaningful title
    • Use the present tense and imperative mood when describing your changes
  • Summarized changes
  • Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
  • This PR is ready to merge and is not Work in Progress.
    • If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.
  • Breaking changes
    • None
    • OR
    • Experimental feature(s) needed
      • Experimental feature name(s):
  • User-facing changes
    • Not Applicable
    • OR
    • Documentation needed
      • Issue filed:
  • Testing - New and feature
    • N/A or can only be tested interactively
    • OR
    • Make sure you've added a new test if existing tests do not effectively test the code changed
  • Tooling
    • I have considered the user experience from a tooling perspective and don't believe tooling will be impacted.
    • OR
    • I have considered the user experience from a tooling perspective and opened an issue in the relevant tool repository. This may include:
      • Impact on PowerShell Editor Services which is used in the PowerShell extension for VSCode
        (which runs in a different PS Host).
        • Issue filed:
      • Impact on Completions (both in the console and in editors) - one of PowerShell's most powerful features.
        • Issue filed:
      • Impact on PSScriptAnalyzer (which provides linting & formatting in the editor extensions).
        • Issue filed:
      • Impact on EditorSyntax (which provides syntax highlighting with in VSCode, GitHub, and many other editors).
        • Issue filed:

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@SteveL-MSFT
Fix Move-Item for FileSystemProvider to use copy-delete instead of …
543c73d
…move for UNC paths
@SteveL-MSFT
Steve Lee (SteveL-MSFT) requested a review from Andrew (anmenaga) as a code owner February 27, 2021 06:18
@ghost ghost assigned Andrew (anmenaga) Feb 27, 2021
@SteveL-MSFT

Steve Lee (SteveL-MSFT) commented Mar 1, 2021

Copy link
Copy Markdown
Member Author

Going to change this to try Move() first, then if AccessDenied, then try CopyDelete().

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@SteveL-MSFT
Change fix to use CopyAndDelete only if AccessDenied error comes up
2513c2f
iSazonov
Ilya (iSazonov) reviewed Mar 2, 2021
View reviewed changes
Comment thread src/System.Management.Automation/namespaces/FileSystemProvider.cs Resolved

Uh oh!

There was an error while loading. Please reload this page.

@SteveL-MSFT
remove unncessary check for Windows if move is to same volume
18d2472
iSazonov
Ilya (iSazonov) approved these changes Mar 2, 2021
View reviewed changes
@iSazonov Ilya (iSazonov) added the CL-General Indicates that a PR should be marked as a general cmdlet change in the Change Log label Mar 2, 2021
@SteveL-MSFT
Steve Lee (SteveL-MSFT) marked this pull request as draft March 2, 2021 19:48
@SteveL-MSFT

Steve Lee (SteveL-MSFT) commented Mar 2, 2021

Copy link
Copy Markdown
Member Author

Making this a draft PR for now, the customer tried a private and still had initial problem after latest change. Trying to determine if they got a different HResult.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov

Ilya (iSazonov) commented Mar 3, 2021

Copy link
Copy Markdown
Collaborator

Looking Directory.MoveTo() implementation https://github.com/dotnet/runtime/blob/79ae74f5ca5c8a6fe3a48935e85bd7374959c570/src/libraries/System.IO.FileSystem/src/System/IO/FileSystem.Windows.cs#L119-L137
I think we can throw for ERROR_PATH_NOT_FOUND and ERROR_ALREADY_EXISTS but fall back for other exceptions.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@SteveL-MSFT

Steve Lee (SteveL-MSFT) commented Mar 3, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

Ilya (@iSazonov) Looks like there can be different HResult for DFS, so can't rely on that. Based on https://github.com/dotnet/runtime/blob/79ae74f5ca5c8a6fe3a48935e85bd7374959c570/src/libraries/System.IO.FileSystem/src/System/IO/FileSystem.Windows.cs#L133, I think we can just try copydelete on an IOException.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@SteveL-MSFT
change to try copydelete on any IOException as different HResult can …
27fb53a
…be returned
@SteveL-MSFT
Steve Lee (SteveL-MSFT) marked this pull request as ready for review March 3, 2021 15:13
@iSazonov

Ilya (iSazonov) commented Mar 3, 2021 •
edited by unfurl-links Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator

If we look GetExceptionForWin32Error()
https://github.com/dotnet/runtime/blob/79ae74f5ca5c8a6fe3a48935e85bd7374959c570/src/libraries/Common/src/System/IO/Win32Marshal.cs#L25
I think we should fall back for IOException and do not fall back for FileNotFoundException, DirectoryNotFoundException, PathTooLongException which is IOException too.

GitHub
dotnet/runtime
.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps. - dotnet/runtime
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov
Ilya (iSazonov) self-requested a review March 3, 2021 17:32
@SteveL-MSFT

Steve Lee (SteveL-MSFT) commented Mar 3, 2021

Copy link
Copy Markdown
Member Author

Ilya (@iSazonov) I see, that makes sense, will update.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Steve Lee (SteveL-MSFT) added 2 commits March 3, 2021 10:19
@SteveL-MSFT
address Ilya's feedback by not falling back on all IOException types
4ea4112
@SteveL-MSFT
update tests
3de7bd2
@SteveL-MSFT

Steve Lee (SteveL-MSFT) commented Mar 3, 2021

Copy link
Copy Markdown
Member Author

Need to think about this some more, the FQErrorID is now different on Linux vs Windows which is not acceptable. Maybe I can find out if there is a limited set of HResult we can check against.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@SteveL-MSFT
Steve Lee (SteveL-MSFT) marked this pull request as draft March 3, 2021 19:56
@iSazonov

Ilya (iSazonov) commented Mar 4, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator

We already discussed this with Rob Holt (@rjmholt) . See https://github.com/PowerShell/PowerShell/pull/13044/files#r446770600

I guess .Net unify the behavior (exceptions) and we can have one code path for all platforms.

I see the test It "Verify Move-Item across devices for directory" is not correct too. It came from from #13044 . I suggest to re-review #13044 now since the code doesn't work and isn't tested correctly neither Windows nor Unix.

GitHub
Fix FS moves across Unix mounts by rjmholt · Pull Request #13044 · PowerShell/PowerShell
PR Summary Fixes #13039. Currently Move-Item does not support cross-mount moves on Unix, because it doesn't check properly whether given paths lie on the same filesystem. This PR fixes the chec...
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@SteveL-MSFT
Unified Unix and Windows paths, check HRESULT for 0x80070005 AccessDe…
5c804aa
…nied for Windows to use CopyAndDelete()
@SteveL-MSFT

Steve Lee (SteveL-MSFT) commented Mar 4, 2021

Copy link
Copy Markdown
Member Author

Ilya (@iSazonov) can you be specific on your concerns about that test?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@SteveL-MSFT
Fix test
4723a93
@SteveL-MSFT
Steve Lee (SteveL-MSFT) marked this pull request as ready for review March 4, 2021 23:11
@SteveL-MSFT
Steve Lee (SteveL-MSFT) requested a review from Rob Holt (rjmholt) March 4, 2021 23:11
@iSazonov

Ilya (iSazonov) commented Mar 5, 2021 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator

Ilya (@iSazonov) can you be specific on your concerns about that test?

I'd expect Move-Item -Path $dir -Destination $destination -ErrorAction Stop throws in the test.

https://github.com/rjmholt/PowerShell/blob/7a2c1fb06cfddff1bb9f2535197154ff143aeefa/test/powershell/Modules/Microsoft.PowerShell.Management/FileSystem.Tests.ps1#L134-L152


And I guess .Net team unified exceptions in the API for all platforms and we could have one code path here. Both return codes fall in IOException and we need only to ignore FileNotFoundException, DirectoryNotFoundException, PathTooLongException.

We must take into account that there are many ways to mount a different file system / volume and they may have different return codes.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@SteveL-MSFT

Steve Lee (SteveL-MSFT) commented Mar 5, 2021

Copy link
Copy Markdown
Member Author

Ilya (@iSazonov) so that test case relies on a testhook to simulate doing a move across devices/volumes. It succeeds because the test hook forces it to throw the exception that results in CopyAndDelete() to be used.

In one of my commits, I tried the approach of ignoring FileNotFoundException, DirectoryNotFoundException, PathTooLongException but that resulted in another FullyQualifiedErrorId to change whereas the current implementation checking for specific HResult which is specific to the Unix error of moving across devices and Windows error of moving across volumes has less impact. I was also concerned that other specific IOExceptions beyond those ignored types would fallback to CopyAndDelete() when it didn't in the past and might cause unintended side effects.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@iSazonov

Ilya (iSazonov) commented Mar 6, 2021

Copy link
Copy Markdown
Collaborator

Ilya (@iSazonov) so that test case relies on a testhook to simulate doing a move across devices/volumes. It succeeds because the test hook forces it to throw the exception that results in CopyAndDelete() to be used.

Sorry, my bad. With -Stop parameter I expect -Should -Not -Throw -Because ""

In one of my commits, I tried the approach of ignoring FileNotFoundException, DirectoryNotFoundException, PathTooLongException but that resulted in another FullyQualifiedErrorId to change whereas the current implementation checking for specific HResult which is specific to the Unix error of moving across devices and Windows error of moving across volumes has less impact. I was also concerned that other specific IOExceptions beyond those ignored types would fallback to CopyAndDelete() when it didn't in the past and might cause unintended side effects.

We had similar discussion in #13634 (comment)
I tried to make minimal changes. But after some conversations we concluded that driver error codes is not reliable way in that context and I made more large refactoring.
So here I also expect that using driver codes is the last thing we should do.
We could think about moving to OneDrive but this works. WebDav? Other Cloud services? Etc.?
It is good if a driver supports moving, but if not? What code will it return? We do not know. I suppose you could always try copying.

My proposal:

        private void MoveDirectoryInfoUnchecked(DirectoryInfo directory, string destinationPath, bool force)
        {
            try
            {
                if (InternalTestHooks.ThrowExdevErrorOnMoveDirectory)
                {
                    throw new IOException("Invalid cross-device link", hresult: TEST_CODE);
                }

                directory.MoveTo(destinationPath);
            }
            catch (Exception exception)
            {
                switch (exception)
                {
                    case FileNotFoundException:
                    case DirectoryNotFoundException:
                    case PathTooLongException:
                        throw;
                    case IOException:
                        CopyAndDelete(directory, destinationPath, force);
                        break;
                    default:
                        throw;
                }
            }
        }
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

anmenaga
Andrew (anmenaga) approved these changes Mar 8, 2021
View reviewed changes

@anmenaga Andrew (anmenaga) left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the current implementation checking for specific HResult which is specific to the Unix error of moving across devices and Windows error of moving across volumes has less impact. I was also concerned that other specific IOExceptions beyond those ignored types would fallback to CopyAndDelete() when it didn't in the past and might cause unintended side effects.

I agree with this.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@anmenaga
Andrew (anmenaga) merged commit 22b3653 into PowerShell:master Mar 8, 2021
@SteveL-MSFT
Steve Lee (SteveL-MSFT) deleted the dfs-move branch March 9, 2021 05:16
@ghost

Deleted user (ghost) commented Mar 16, 2021

Copy link
Copy Markdown

🎉v7.2.0-preview.4 has been released which incorporates this pull request.:tada:

Handy links:

  • Release Notes
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hazychill hazychill mentioned this pull request Mar 20, 2021
Move-Item: Move directory to another drive fails #15072
Closed
@SteveL-MSFT Steve Lee (SteveL-MSFT) mentioned this pull request Mar 21, 2021
Fallback to CopyAndDelete() when MoveTo() fails due to an IOException #15077
Merged
8 of 22 tasks
@iSazonov Ilya (iSazonov) mentioned this pull request Aug 26, 2021
Recent Move-Item changes causing Windows platform regressions #16000
Closed
5 tasks done
Thatgfsj (Thatgfsj) pushed a commit to Thatgfsj-contribs/PowerShell that referenced this pull request Aug 6, 2026
@SteveL-MSFT
Fix Move-Item for FileSystemProvider to use copy-delete instead of …
8be7003
…move for DFS paths (PowerShell#14913)
@coderabbitai coderabbitai Bot mentioned this pull request Aug 14, 2026
feat(release): publish checksums and verify them before installing veryfront/veryfront-code#3707
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@iSazonov Ilya (iSazonov) Awaiting requested review from iSazonov
@rjmholt Rob Holt (rjmholt) Awaiting requested review from rjmholt
1 more reviewer
@anmenaga Andrew (anmenaga) anmenaga approved these changes
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@anmenaga Andrew (anmenaga)

Labels

CL-General Indicates that a PR should be marked as a general cmdlet change in the Change Log

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@SteveL-MSFT @iSazonov @anmenaga
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.