Merge pull request #57637 from JoeRobich/update-to-net6.0GA

Use the .NET 6.0 SDK for building (Arcade7)

docs/contributing/Building, Debugging, and Testing on Unix.md

@@ -1,6 +1,6 @@
 # Building, Debugging and Testing on Unix
-This guide is meant to help developers setup an environment for debugging / contributing to Roslyn from Linux. 
-Particularly for developers who aren't experienced with .NET Core development on Linux. 
+This guide is meant to help developers setup an environment for debugging / contributing to Roslyn from Linux.
+Particularly for developers who aren't experienced with .NET Core development on Linux.
 
 ## Working with the code
 1. Ensure the commands `git` and `curl` are available
@@ -12,7 +12,7 @@ Particularly for developers who aren't experienced with .NET Core development on
 1. Install [VS Code](https://code.visualstudio.com/Download)
     - After you install VS Code, install the [C# extension](https://marketplace.visualstudio.com/items?itemName=ms-dotnettools.csharp)
     - Important tip: You can look up editor commands by name by hitting *Ctrl+Shift+P*, or by hitting *Ctrl+P* and typing a `>` character. This will help you get familiar with editor commands mentioned below. On a Mac, use *⌘* instead of *Ctrl*.
-2. Install the [.NET 6.0 RC 2 SDK](https://dotnet.microsoft.com/download/dotnet-core/6.0).
+2. Install the [.NET 6.0 SDK](https://dotnet.microsoft.com/download/dotnet-core/6.0).
 3. You can build from VS Code by running the *Run Build Task* command, then selecting an appropriate task such as *build* or *build current project* (the latter builds the containing project for the current file you're viewing in the editor).
 4. You can run tests from VS Code by opening a test class in the editor, then using the *Run Tests in Context* and *Debug Tests in Context* editor commands. You may want to bind these commands to keyboard shortcuts that match their Visual Studio equivalents (**Ctrl+R, T** for *Run Tests in Context* and **Ctrl+R, Ctrl+T** for *Debug Tests in Context*).
 
@@ -23,14 +23,14 @@ To run all tests in a single project, it's recommended to use the `dotnet test p
 
 ## GitHub
 The best way to clone and push is to use SSH. On Windows you typically use HTTPS and this is not directly compatible
-with two factor authentication (requires a PAT). The SSH setup is much simpler and GitHub has a great HOWTO for 
+with two factor authentication (requires a PAT). The SSH setup is much simpler and GitHub has a great HOWTO for
 getting this setup.
 
 https://help.github.com/articles/connecting-to-github-with-ssh/
 
 ## Debugging test failures
 The best way to debug is using lldb with the SOS plugin. This is the same SOS as used in WinDbg and if you're familiar
-with it then lldb debugging will be pretty straight forward. 
+with it then lldb debugging will be pretty straight forward.
 
 The [dotnet/diagnostics](https://github.com/dotnet/diagnostics) repo has more information:
 
@@ -45,10 +45,10 @@ CoreCLR also has some guidelines for specific Linux debugging scenarios:
 
 Corrections:
 - LLDB and createdump must be run as root
-- `dotnet tool install -g dotnet-symbol` must be run from `$HOME` 
+- `dotnet tool install -g dotnet-symbol` must be run from `$HOME`
 
 ### Core Dumps
-The CoreClr does not used the standard core dumping mechanisms on Linux. Instead you must specify via 
+The CoreClr does not used the standard core dumping mechanisms on Linux. Instead you must specify via
 environment variables that you want a core dump to be produced. The simplest setup is to do the following:
 
 ```
@@ -79,12 +79,12 @@ indeed crashing in the GC.
 Note: this variables can also be used on Windows as well.
 
 ## Ubuntu 18.04
-The recommended OS for developing Roslyn is Ubuntu 18.04. This guide was written using Ubuntu 18.04 but should be 
+The recommended OS for developing Roslyn is Ubuntu 18.04. This guide was written using Ubuntu 18.04 but should be
 applicable to most Linux environments. Ubuntu 18.04 was chosen here due to it's support for enhanced VMs in Hyper-V.
 This makes it easier to use from a Windows machine: full screen, copy / paste, etc ...
 
 ### Hyper-V
-Hyper-V has a builtin Ubuntu 18.04 image which supports enhanced mode. Here is a tutorial for creating 
+Hyper-V has a builtin Ubuntu 18.04 image which supports enhanced mode. Here is a tutorial for creating
 such an image:
 
 https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/quick-create-virtual-machine
@@ -93,10 +93,10 @@ When following this make sure to:
 1. Click Installation Source and uncheck "Windows Secure Boot"
 1. Complete the Ubuntu installation wizard. Full screen mode won't be available until this is done.
 
-Overall this takes about 5-10 minutes to complete. 
+Overall this takes about 5-10 minutes to complete.
 
 ### Source Link
-Many of the repositories that need to be built use source link and it crashes on Ubuntu 18.04 due to dependency changes. 
+Many of the repositories that need to be built use source link and it crashes on Ubuntu 18.04 due to dependency changes.
 To disable source link add the following to the `Directory.Build.props` file in the root of the repository.
 
 ``` xml
@@ -109,6 +109,6 @@ To disable source link add the following to the `Directory.Build.props` file in
 Make sure to install the following via `apt install`
 
 - clang
-- lldb 
+- lldb
 - cmake
 - xrdp

docs/contributing/Building, Debugging, and Testing on Windows.md

@@ -20,7 +20,7 @@ The minimal required version of .NET Framework is 4.7.2.
     - Ensure Visual Studio is on Version "17.0" or greater
     - Ensure "Use previews of the .NET Core SDK" is checked in Tools -> Options -> Environment -> Preview Features
     - Restart Visual Studio
-1. [.NET 6.0 RC 2 SDK](https://dotnet.microsoft.com/download/dotnet-core/6.0) [Windows x64 installer](https://dotnet.microsoft.com/download/dotnet/thank-you/sdk-6.0.100-rc.2-windows-x64-installer)
+1. [.NET 6.0 SDK](https://dotnet.microsoft.com/download/dotnet-core/6.0) [Windows x64 installer](https://dotnet.microsoft.com/download/dotnet/thank-you/sdk-6.0.100-windows-x64-installer)
 1. [PowerShell 5.0 or newer](https://docs.microsoft.com/en-us/powershell/scripting/setup/installing-windows-powershell). If you are on Windows 10, you are fine; you'll only need to upgrade if you're on earlier versions of Windows. The download link is under the ["Upgrading existing Windows PowerShell"](https://docs.microsoft.com/en-us/powershell/scripting/install/installing-windows-powershell?view=powershell-6#upgrading-existing-windows-powershell) heading.
 1. Run Restore.cmd
 1. Open Roslyn.sln

eng/Version.Details.xml

@@ -13,18 +13,18 @@
     </Dependency>
   </ProductDependencies>
   <ToolsetDependencies>
-    <Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="7.0.0-beta.21474.2">
+    <Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="7.0.0-beta.21566.10">
       <Uri>https://github.com/dotnet/arcade</Uri>
-      <Sha>3ea0d860c6973f2cbadc9e895c7ec2cbdaec4ad5</Sha>
+      <Sha>53cc1bc2e555aa7aea95884575d22e21d63708cf</Sha>
       <SourceBuild RepoName="arcade" ManagedOnly="true" />
     </Dependency>
     <Dependency Name="Microsoft.Net.Compilers.Toolset" Version="4.0.0-5.21469.2">
       <Uri>https://github.com/dotnet/roslyn</Uri>
       <Sha>c1d8c6f043bc80425c6828455eb57f8a404759c6</Sha>
     </Dependency>
-    <Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="7.0.0-beta.21474.2">
+    <Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="7.0.0-beta.21566.10">
       <Uri>https://github.com/dotnet/arcade</Uri>
-      <Sha>3ea0d860c6973f2cbadc9e895c7ec2cbdaec4ad5</Sha>
+      <Sha>53cc1bc2e555aa7aea95884575d22e21d63708cf</Sha>
     </Dependency>
   </ToolsetDependencies>
 </Dependencies>

eng/common/msbuild.ps1

@@ -6,6 +6,7 @@ Param(
   [switch] $ci,
   [switch] $prepareMachine,
   [switch] $excludePrereleaseVS,
+  [string] $msbuildEngine = $null,
   [Parameter(ValueFromRemainingArguments=$true)][String[]]$extraArgs
 )
 

eng/common/native/init-compiler.sh

@@ -3,23 +3,25 @@
 # This file detects the C/C++ compiler and exports it to the CC/CXX environment variables
 #
 
-if [[ "$#" -lt 2 ]]; then
+if [[ "$#" -lt 3 ]]; then
   echo "Usage..."
-  echo "init-compiler.sh <Architecture> <compiler> <compiler major version> <compiler minor version>"
+  echo "init-compiler.sh <script directory> <Architecture> <compiler> <compiler major version> <compiler minor version>"
+  echo "Specify the script directory."
   echo "Specify the target architecture."
   echo "Specify the name of compiler (clang or gcc)."
   echo "Specify the major version of compiler."
   echo "Specify the minor version of compiler."
   exit 1
 fi
 
-. "$( cd -P "$( dirname "$0" )" && pwd )"/../pipeline-logging-functions.sh
-
-build_arch="$1"
-compiler="$2"
+nativescriptroot="$1"
+build_arch="$2"
+compiler="$3"
 cxxCompiler="$compiler++"
-majorVersion="$3"
-minorVersion="$4"
+majorVersion="$4"
+minorVersion="$5"
+
+. "$nativescriptroot"/../pipeline-logging-functions.sh
 
 # clear the existing CC and CXX from environment
 CC=
@@ -48,8 +50,8 @@ if [[ -z "$CLR_CC" ]]; then
     # Set default versions
     if [[ -z "$majorVersion" ]]; then
         # note: gcc (all versions) and clang versions higher than 6 do not have minor version in file name, if it is zero.
-        if [[ "$compiler" == "clang" ]]; then versions=( 12 11 10 9 8 7 6.0 5.0 4.0 3.9 3.8 3.7 3.6 3.5 )
-        elif [[ "$compiler" == "gcc" ]]; then versions=( 11 10 9 8 7 6 5 4.9 ); fi
+        if [[ "$compiler" == "clang" ]]; then versions=( 13 12 11 10 9 8 7 6.0 5.0 4.0 3.9 3.8 3.7 3.6 3.5 )
+        elif [[ "$compiler" == "gcc" ]]; then versions=( 12 11 10 9 8 7 6 5 4.9 ); fi
 
         for version in "${versions[@]}"; do
             parts=(${version//./ })

eng/common/post-build/sourcelink-validation.ps1

@@ -22,6 +22,11 @@ $RetryWaitTimeInSeconds = 30
 # Wait time between check for system load
 $SecondsBetweenLoadChecks = 10
 
+if (!$InputPath -or !(Test-Path $InputPath)){
+  Write-Host "No files to validate."
+  ExitWithExitCode 0
+}
+
 $ValidatePackage = {
   param( 
     [string] $PackagePath                                 # Full path to a Symbols.NuGet package

eng/common/post-build/symbols-validation.ps1

@@ -4,9 +4,11 @@ param(
   [Parameter(Mandatory = $true)][string] $DotnetSymbolVersion, # Version of dotnet symbol to use
   [Parameter(Mandatory = $false)][switch] $CheckForWindowsPdbs, # If we should check for the existence of windows pdbs in addition to portable PDBs
   [Parameter(Mandatory = $false)][switch] $ContinueOnError, # If we should keep checking symbols after an error
-  [Parameter(Mandatory = $false)][switch] $Clean                  # Clean extracted symbols directory after checking symbols
+  [Parameter(Mandatory = $false)][switch] $Clean,           # Clean extracted symbols directory after checking symbols
+  [Parameter(Mandatory = $false)][string] $SymbolExclusionFile  # Exclude the symbols in the file from publishing to symbol server
 )
 
+. $PSScriptRoot\..\tools.ps1
 # Maximum number of jobs to run in parallel
 $MaxParallelJobs = 16
 
@@ -25,14 +27,28 @@ if ($CheckForWindowsPdbs) {
   $WindowsPdbVerificationParam = "--windows-pdbs"
 }
 
+$ExclusionSet = New-Object System.Collections.Generic.HashSet[string];
+
+if (!$InputPath -or !(Test-Path $InputPath)){
+  Write-Host "No symbols to validate."
+  ExitWithExitCode 0
+}
+
+#Check if the path exists
+if ($SymbolExclusionFile -and (Test-Path $SymbolExclusionFile)){
+  [string[]]$Exclusions = Get-Content "$SymbolExclusionFile"
+  $Exclusions | foreach { if($_ -and $_.Trim()){$ExclusionSet.Add($_)} }
+}
+else{
+  Write-Host "Symbol Exclusion file does not exists. No symbols to exclude."
+}
+
 $CountMissingSymbols = {
   param( 
     [string] $PackagePath, # Path to a NuGet package
     [string] $WindowsPdbVerificationParam # If we should check for the existence of windows pdbs in addition to portable PDBs
   )
 
-  . $using:PSScriptRoot\..\tools.ps1
-
   Add-Type -AssemblyName System.IO.Compression.FileSystem
 
   Write-Host "Validating $PackagePath "
@@ -118,17 +134,17 @@ $CountMissingSymbols = {
         # Save the output and get diagnostic output
         $output = & $dotnetSymbolExe --symbols --modules $WindowsPdbVerificationParam $TargetServerParam $FullPath -o $SymbolsPath --diagnostics | Out-String
 
-        if (Test-Path $PdbPath) {
-          return 'PDB'
+        if ((Test-Path $PdbPath) -and (Test-path $SymbolPath)) {
+          return 'Module and PDB for Module'
         }
-        elseif (Test-Path $NGenPdb) {
-          return 'NGen PDB'
+        elseif ((Test-Path $NGenPdb) -and (Test-Path $PdbPath) -and (Test-Path $SymbolPath)) {
+          return 'Dll, PDB and NGen PDB'
         }
-        elseif (Test-Path $SODbg) {
-          return 'DBG for SO'
+        elseif ((Test-Path $SODbg) -and (Test-Path $SymbolPath)) {
+          return 'So and DBG for SO'
         }  
-        elseif (Test-Path $DylibDwarf) {
-          return 'Dwarf for Dylib'
+        elseif ((Test-Path $DylibDwarf) -and (Test-Path $SymbolPath)) {
+          return 'Dylib and Dwarf for Dylib'
         }  
         elseif (Test-Path $SymbolPath) {
           return 'Module'
@@ -142,37 +158,44 @@ $CountMissingSymbols = {
       return $null
     }
 
-    $FileGuid = New-Guid
-    $ExpandedSymbolsPath = Join-Path -Path $SymbolsPath -ChildPath $FileGuid
-
-    $SymbolsOnMSDL = & $FirstMatchingSymbolDescriptionOrDefault `
-        -FullPath $FileName `
-        -TargetServerParam '--microsoft-symbol-server' `
-        -SymbolsPath "$ExpandedSymbolsPath-msdl" `
-        -WindowsPdbVerificationParam $WindowsPdbVerificationParam
-    $SymbolsOnSymWeb = & $FirstMatchingSymbolDescriptionOrDefault `
-        -FullPath $FileName `
-        -TargetServerParam '--internal-server' `
-        -SymbolsPath "$ExpandedSymbolsPath-symweb" `
-        -WindowsPdbVerificationParam $WindowsPdbVerificationParam
-
-    Write-Host -NoNewLine "`t Checking file " $FileName "... "
-
-    if ($SymbolsOnMSDL -ne $null -and $SymbolsOnSymWeb -ne $null) {
-      Write-Host "Symbols found on MSDL ($SymbolsOnMSDL) and SymWeb ($SymbolsOnSymWeb)"
+    $FileRelativePath = $FileName.Replace("$ExtractPath\", "")
+    if (($($using:ExclusionSet) -ne $null) -and ($($using:ExclusionSet).Contains($FileRelativePath) -or ($($using:ExclusionSet).Contains($FileRelativePath.Replace("\", "/"))))){
+      Write-Host "Skipping $FileName from symbol validation"
     }
-    else {
-      $MissingSymbols++
 
-      if ($SymbolsOnMSDL -eq $null -and $SymbolsOnSymWeb -eq $null) {
-        Write-Host 'No symbols found on MSDL or SymWeb!'
+    else {
+      $FileGuid = New-Guid
+      $ExpandedSymbolsPath = Join-Path -Path $SymbolsPath -ChildPath $FileGuid
+
+      $SymbolsOnMSDL = & $FirstMatchingSymbolDescriptionOrDefault `
+          -FullPath $FileName `
+          -TargetServerParam '--microsoft-symbol-server' `
+          -SymbolsPath "$ExpandedSymbolsPath-msdl" `
+          -WindowsPdbVerificationParam $WindowsPdbVerificationParam
+      $SymbolsOnSymWeb = & $FirstMatchingSymbolDescriptionOrDefault `
+          -FullPath $FileName `
+          -TargetServerParam '--internal-server' `
+          -SymbolsPath "$ExpandedSymbolsPath-symweb" `
+          -WindowsPdbVerificationParam $WindowsPdbVerificationParam
+
+      Write-Host -NoNewLine "`t Checking file " $FileName "... "
+
+      if ($SymbolsOnMSDL -ne $null -and $SymbolsOnSymWeb -ne $null) {
+        Write-Host "Symbols found on MSDL ($SymbolsOnMSDL) and SymWeb ($SymbolsOnSymWeb)"
       }
       else {
-        if ($SymbolsOnMSDL -eq $null) {
-          Write-Host 'No symbols found on MSDL!'
+        $MissingSymbols++
+
+        if ($SymbolsOnMSDL -eq $null -and $SymbolsOnSymWeb -eq $null) {
+          Write-Host 'No symbols found on MSDL or SymWeb!'
         }
         else {
-          Write-Host 'No symbols found on SymWeb!'
+          if ($SymbolsOnMSDL -eq $null) {
+            Write-Host 'No symbols found on MSDL!'
+          }
+          else {
+            Write-Host 'No symbols found on SymWeb!'
+          }
         }
       }
     }

eng/common/sdl/configure-sdl-tool.ps1

@@ -69,13 +69,13 @@ try {
     # For some tools, add default and automatic args.
     if ($tool.Name -eq 'credscan') {
       if ($targetDirectory) {
-        $tool.Args += "TargetDirectory < $TargetDirectory"
+        $tool.Args += "`"TargetDirectory < $TargetDirectory`""
       }
-      $tool.Args += "OutputType < pre"
+      $tool.Args += "`"OutputType < pre`""
       $tool.Args += $CrScanAdditionalRunConfigParams
     } elseif ($tool.Name -eq 'policheck') {
       if ($targetDirectory) {
-        $tool.Args += "Target < $TargetDirectory"
+        $tool.Args += "`"Target < $TargetDirectory`""
       }
       $tool.Args += $PoliCheckAdditionalRunConfigParams
     }

eng/common/sdl/execute-all-sdl-tools.ps1

@@ -124,7 +124,7 @@ try {
   Exec-BlockVerbosely {
     & $(Join-Path $PSScriptRoot 'run-sdl.ps1') `
       -GuardianCliLocation $guardianCliLocation `
-      -WorkingDirectory $workingDirectory `
+      -WorkingDirectory $SourceDirectory `
       -UpdateBaseline $UpdateBaseline `
       -GdnFolder $gdnFolder
   }

eng/common/templates/job/execute-sdl.yml

@@ -60,11 +60,7 @@ jobs:
     - name: GuardianPackagesConfigFile
       value: $(Build.SourcesDirectory)\eng\common\sdl\packages.config
   pool:
-    # To extract archives (.tar.gz, .zip), we need access to "tar", added in Windows 10/2019.
-    ${{ if eq(parameters.extractArchiveArtifacts, 'false') }}:
-      name: Hosted VS2017
-    ${{ if ne(parameters.extractArchiveArtifacts, 'false') }}:
-      vmImage: windows-2019
+    vmImage: windows-2019
   steps:
   - checkout: self
     clean: true

eng/common/templates/job/onelocbuild.yml

@@ -4,14 +4,15 @@ parameters:
 
   # Optional: A defined YAML pool - https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=vsts&tabs=schema#pool
   pool:
-    vmImage: vs2017-win2016
+    vmImage: 'windows-2019'
 
   CeapexPat: $(dn-bot-ceapex-package-r) # PAT for the loc AzDO instance https://dev.azure.com/ceapex
   GithubPat: $(BotAccount-dotnet-bot-repo-PAT)
 
   SourcesDirectory: $(Build.SourcesDirectory)
   CreatePr: true
   AutoCompletePr: false
+  ReusePr: true
   UseLfLineEndings: true
   UseCheckedInLocProjectJson: false
   LanguageSet: VS_Main_Languages
@@ -64,6 +65,8 @@ jobs:
         ${{ if eq(parameters.CreatePr, true) }}:
           isAutoCompletePrSelected: ${{ parameters.AutoCompletePr }}
           isUseLfLineEndingsSelected: ${{ parameters.UseLfLineEndings }}
+          ${{ if eq(parameters.RepoType, 'gitHub') }}:
+            isShouldReusePrSelected: ${{ parameters.ReusePr }}
         packageSourceAuth: patAuth
         patVariable: ${{ parameters.CeapexPat }}
         ${{ if eq(parameters.RepoType, 'gitHub') }}: