Symptoms
Overall slowness of K2 applications accompanied by high CPU usage on K2 sever. K2 host server log shows the following exceptions:
SourceCode.Diagnostics.Log:
Failed to log entry:
Exception=System.IO.IOException:
The requested operation could not be completed due to a file system limitation
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.WriteCore(Byter] buffer, Int32 offset, Int32 count)
at System.IO.FileStream.FlushWrite(Boolean calledFromFinalizer)
at System.IO.FileStream.Dispose(Boolean disposing)
at System.IO.Stream.Close()
at System.IO.StreamWriter.Dispose(Boolean disposing)
at System.IO.TextWriter.Dispose()
at System.IO.File.InternalAppendAllText(String path, String contents, Encodin
g encoding)
at SourceCode.Diagnostics.Log.LogWorker(Object state)
SourceCode.Diagnostics.Log: Failed to log entry: Category=SourceCode.Diagnostics
, Message=
YYYY-MM-DD THH:MM:SS SourceCode.Diagnostics: OnFirstChanceException
Exception:
System.IO.IOException: The requested operation could not be completed due to a file system limitation
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
StackTrace:
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.WriteCore(ByteC] buffer, Int32 offset, Int32 count)
at System.IO.FileStream.FlushWrite(Boolean calledFromFinalizer)
at System.IO.FileStream.Flush(Boolean flushToDisk)
at System.IO.StreamWriter.Dispose(Boolean disposing)
at System.IO.TextWriter.Dispose()
at System.IO.File.InternalAppendAllText(String path, String contents, Encodin
g encoding)
at SourceCode.Diagnostics.Log.LogWorker(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionCo
ntext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, C
ontextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
**************
Diagnoses
Based on exception details it is possible to say that it is caused by abnormally big log file which cannot be appended by new data due to its size: "SourceCode.Diagnostics.Log: Failed to log entry: Exception=System.IO.IOException: The requested operation could not be completed due to a file system limitation"
In this case error message indicates log file name - SourceCode.Diagnostics.Log, and you can further confirm this by using this PowerShell script:
Get-ChildItem -Path 'C:Program Files (x86)K2 blackpearl' -Recurse -Force -File | Select-Object -Property FullName,@{Name='SizeGB'Expression={$_.Length / 1GB}},@{Name='SizeMB'Expression={$_.Length / 1MB}} | Sort-Object { $_.SizeGB } -Descending | Out-GridView
From the output of this script it is possible to see which specific log file located in "'C:Program Files (x86)K2 blackpearl" folder is abnormally big.
In this specific case it was SourceCode.Diagnostics.Log file which was located in "'C:Program Files (x86)K2 blackpearlHostServerBinHostServersInternal" folder and had a size of about 25 GB. This file is not a part of standard K2 logging and was generated by debug assembly which was installed while investigating another issue.
As usual any debug assemblies supposed to be removed after investigation/troubleshooting activities complete.
Resolution
Read exception details carefully to see if it indicates any specific log file which causes this exception. In general this exception could be caused by excessively large size of ADUM logs (in case you have not set maximum file size limit which is in place by default), SmO logs (those do not have configurable maximum file size limit) or logs generated by debug assemblies.
Always switch off debug logging once troubleshooting/investigation of specific issue complete and do not leave debug logging enabled for prolonged periods of time unless it is necessary for investigation of specific problem.
