Some Notes on the Machines

The hidden NTFS “System Volume Information” folders on Windows machines, can build up and up and up in size. I’ve seen instances ranging from 20G to hundreds of gigabytes, and every time this occurs, the overall system slows down, and often slows down a whole lot. SpaceSniffer is my favorite method of identifying this situation, but there are many. The only preventative I have been able to identify so far, is here.

SentinelOne (S1) considers deletion of volume shadows to be bad actor behavior, because it often is a way that cryptolockers delete last-known-good checkpoints. In order to clear a huge SVI folder with S1 installed, one must do this:

1. First get the Passphrase for the machine, from the S1 console. It’s under Actions, you can choose Show Passphrase. Do be aware that your S1 admin will probably receive a notice that you have asked for this.
2. cd "C:\Program Files\SentinelOne\Sentinel*"
3. Please put the actual passphrase in, and the quotes are necessary:
   .\sentinelctl.exe unload -slam -k "<passphrase>"
4. vssadmin delete shadows /all
5. Only if this is a server, check System Volume Information size again. If it’s still big, we have two options.
   1. The first is to do these two:
      diskshadow
      (within diskshadow’s command line:) delete shadows all
      
      This can take a while, especially if SVI is big, e.g., more than 20-30 gigabytes. It can get huge occasionally, hundreds of gigabytes. I recently saw 1,022 shadow copies deleted (it tells you the count at the end) from one server.
   2. The second is to do this:
      wmic shadowcopy delete /nointeractive
      which runs the cleanup nicely, and possibly a bit faster than the other.

6. .\sentinelctl.exe load -slam

And you are done.

If you should need to reenable S1 after work such as the above, here’s a paste:

cd "C:\Program Files\SentinelOne\Sentinel*"
.\sentinelctl.exe load -slam

The overall cause of buildup of orphan shadows in “System Volume Information” folders, is storage not being able to keep up with demand. If the situation is bad enough, this won’t help much, but it has helped in many instances. It’s three registry entries. Here’s Powershell code to get them in and engaged.

$NewMaxShadowCopies = 8
$NewMinDiffAreaFileSize = 128

# http://www.tomsitpro.com/articles/powershell_registry-powershell_command_line,2-152.html

function setupDWORD {
    param( [string]$regPath, [string]$nameForDWORD, [long]$valueForDWORD )

    ##############
    # Error out if cannot touch the registry area at all
    If ( !(Test-Path $regPath) ) {
        Try {
            New-Item $regPath -Force -ErrorAction SilentlyContinue
            }
        Catch {
            Write-Error ("Could not visit or create registry path " + $regPath)
            Return
            }
        }

    #############
    # If an existing registry entry exists, store its value to report later
    Try {
        $oldValueProperty = Get-ItemProperty -Path $regPath -Name $nameForDWORD -ErrorAction SilentlyContinue
        $oldValue = $oldValueProperty.$nameforDWORD
        }
    Catch {
        $oldValue = ""
        }

    #############
    # Report the changes to make
    Write-Output ("DWORD to write: " + $nameForDWORD)
    Write-Output ("at registry path " + $regPath)
    If ($oldValue -ne "") {
        Write-Output ("Original value is " + $oldValue)
        }
    else {
        Write-Output "No original present."
        }
    Write-Output ("New value is " + $valueforDWORD)

    ############
    # Report no changes to make, set new registry entry, or error out
	If ($oldValue -eq $valueforDWORD) {
		Write-Output "No change to make."
		""
		Return
		}
    Try {
        New-ItemProperty -Path $regPath -Name $nameForDWORD -Value $valueForDWORD -PropertyType DWORD -Force -ErrorAction SilentlyContinue > $null
        }
    Catch {
        Write-Error "Failed!"
        ""
        Return
        }

    "Succeeded!"
    ""
    }

setupDWORD "HKLM:\System\CurrentControlSet\Services\VSS\Settings" "MaxShadowCopies" $NewMaxShadowCopies

setupDWORD "HKLM:\System\CurrentControlSet\Services\VolSnap" "MinDiffAreaFileSize" $NewMinDiffAreaFileSize

setupDWORD 'HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows' 'DeleteStaleTaskCache' 1

""

"Restarting VSS..."

Restart-Service -Force -Name "VSS"

""

"Complete!"
""

This is a great way to get things removed where nothing else works.

learn.microsoft.com/en-us/sysinternals/downloads/pendmoves

Often, the QuickBooks desktop installer will not accept the serial number and/or product number, the fields are too short. QB support pages advise changing font scaling, but often this does not help. The best workaround so far, has been to copy the serial and product numbers to Notepad, then paste them in all at once, rather than trying to type them in directly.

For a long time the standard was, contact your S1 support and receive a removal tool. I don’t know if this always works, but it did for one corrupt agent:

SentinelOneInstaller_windows_64bit_v22_2_4_558.exe --clean_only --dont_preserve_config_dir --dont_preserve_agent_uid -t xyzpdqxyzpdq

where xyzpdqxyzpdq is the site token (much longer of course). Not entirely sure the -t xyzpdqxyzpdq is needed.

The most reliable way I had for a long time, was using a scheduled task. But Powershell for this changes from Windows version to Windows version. Here’s a new method, it uses the time specification in the ‘shutdown’ command, to reboot the machine tomorrow at 3AM:

$tomorrow3AM = (Get-Date).AddHours(24)
$tomorrow3AM = $tomorrow3AM.AddHours( ($tomorrow3AM.Hour * -1) + 3 )
$tomorrow3AM = $tomorrow3AM.AddMinutes( $tomorrow3AM.Minute * -1 )
$tomorrow3AM = $tomorrow3AM.AddSeconds( $tomorrow3AM.Second * -1 )
$tomorrow3AM = $tomorrow3AM.AddMilliseconds( $tomorrow3AM.Millisecond * -1 )
$SecondsFromNow = ($tomorrow3AM - (Get-Date)).TotalSeconds
shutdown -f -r -t ([int] $SecondsFromNow)

Remember that huge text file with all of the DHCP reservations, load average, and everything else? Well, go to:

-Monitor -> Devices
-Choose a firewall
-Go to Live Status -> Diagnostic Tools
-Choose the Snapshot tab.
-Click to download a snapshot file.

Then, in that file:

Fireware_support.tgz\
Fireware_support.tar\
Fireware_XTM_Support.tgz\
Fireware_XTM_Support.tar\
support\system\system_status.txt

Sometimes, when one logs into https://portal.office.com, the link/button for downloading is not present at the upper-right corner. One thing to try is this, right after login:

https://portal.office.com/account/?ref=MeControl

If that doesn’t get you to downloads, try these steps:

Click on ID ball at the upper right to pull down menu
Click on View Account
Click on Subscriptions at lower left
Click on Apps & devices near lower left

Recently it was discovered that a Windows server was running very slow because the Security log’s maximum size was set to 40 gigabytes. Here is a Powershell bit which will look at all event logs, set their max size to 2.5M if set larger, and clear them. Seems to free up a nice healthy dollop of performance in general.

wevtutil el | Foreach-Object {
	$LogObject = Get-WinEvent -ListLog $_
	If ( $LogObject.MaximumSizeInBytes -gt 2500KB ) {
		"$_ has max set to larger than 2.5M.  Setting to 2.5M."
		$LogObject.MaximumSizeInBytes = 2500KB
		$LogObject.SaveChanges()
		}
	wevtutil cl $_
	"$_ cleared."
}

There are times when an operation, a software install or configure perhaps, will error with “Cannot open log for source ———-. You may not have write access.” The below will do the above, and also give read/write to every local admin.

wevtutil el | Foreach-Object {
	wevtutil sl $_ "/ca:O:BAG:SYD:(A;;0x1;;;SY)(A;;0x5;;;BA)(A;;0x1;;;LA)(A;;0x3;;;LA)"
	$LogObject = Get-WinEvent -ListLog $_
	If ( $LogObject.MaximumSizeInBytes -gt 2500KB ) {
		"$_ has max set to larger than 2.5M.  Setting to 2.5M."
		$LogObject.MaximumSizeInBytes = 2500KB
		$LogObject.SaveChanges()
		}
	wevtutil cl $_
	"$_ cleared."
}

The security string is written in something called SDDL. Some more info:

https://itconnect.uw.edu/wares/msinf/other-help/understanding-sddl-syntax/

Microsoft is loading Windows with gaming bits and other things which take resources. The last two builds of 10, and 11, have ‘winget’, which make it quite easy to remove things:

winget uninstall "Xbox TCUI"
winget uninstall "Xbox Console Companion"
winget uninstall "Xbox Game Bar Plugin"
winget uninstall "Xbox Game Bar"
winget uninstall "Xbox Identity Provider"
winget uninstall "Xbox Game Speech Window"
winget uninstall "Phone Link"
winget uninstall "Movies & TV"