Ended up creating my own resolution here.
Ran on test-environment first and nothing seems to break.
Added the following powershell as a scheduled task. The script deletes all feilas and empty folder older than 150 days in the specified folder.
$limit = (Get-Date).AddDays(-150)
$path = "PATH_TO_THE TEMP_FOLDER"
# Delete files older than the $limit.
Get-ChildItem -Path $path -Recurse -Force | Where-Object { !$_.PSIsContainer -and $_.CreationTime -lt $limit } | Remove-Item -Force
# Delete any empty directories left behind after deleting the old files.
Get-ChildItem -Path $path -Recurse -Force | Where-Object { $_.PSIsContainer -and (Get-ChildItem -Path $_.FullName -Recurse -Force | Where-Object { !$_.PSIsContainer }) -eq $null } | Remove-Item -Force -Recurse