Hey!
I have a few logs containing million+ lines of logs, all with date stamps and need to find out how long the machine was running with those logs. Now in theory this is really easy, I just use DateTime.ParseExact and subtract the first and last date and voila, BUT!
The time stamps are in 12hour format and conveniently don't have AM or PM :)
So right now If I do startdate - end date, the accuracy of my results could vary by 24 hours. I want to get this as exact as possible.
There are more fun variables that are added to this mix, but I will leave it at that for now :P
I was thinking I could use a for loop, check when the day changes, if it changes then check how many logs are in there for that day if the logs show less than 12 hours then we know that the start time was in the PM. And I could do the same for the end?
Only thing that worries me is that these logs are million + lines, so not sure how the performance of this will be, as it is it takes a bit of time using the "Readalllines" function.
Any feedback is greatly appreciated :)
Here is my code right now:
I have a few logs containing million+ lines of logs, all with date stamps and need to find out how long the machine was running with those logs. Now in theory this is really easy, I just use DateTime.ParseExact and subtract the first and last date and voila, BUT!
The time stamps are in 12hour format and conveniently don't have AM or PM :)
So right now If I do startdate - end date, the accuracy of my results could vary by 24 hours. I want to get this as exact as possible.
There are more fun variables that are added to this mix, but I will leave it at that for now :P
I was thinking I could use a for loop, check when the day changes, if it changes then check how many logs are in there for that day if the logs show less than 12 hours then we know that the start time was in the PM. And I could do the same for the end?
Only thing that worries me is that these logs are million + lines, so not sure how the performance of this will be, as it is it takes a bit of time using the "Readalllines" function.
Any feedback is greatly appreciated :)
Here is my code right now:
Code:
Public Class Form1
Private Sub btnfolder_Click(sender As System.Object, e As System.EventArgs) Handles btnfolder.Click
Dim folder As String = ""
Dim counter As Integer = 0
FolderBrowserDialog1.ShowDialog()
folder = FolderBrowserDialog1.SelectedPath
Dim format As String = ""
Dim starttime As Date
Dim endtime As Date
Dim all_lines() As String
Dim delimiter As String = "|"
Dim filenames As String() = Directory.GetFiles(folder, "*.csv")
Dim outputfile As String = folder & "\Report.csv"
Dim output As StreamWriter = New StreamWriter(outputfile)
Dim totaltime As Integer = 0
ProgressBar1.Value = 0
ProgressBar1.Maximum = filenames.Length + 1
For Each Files In filenames
ProgressBar1.Value = ProgressBar1.Value + 1
If Files.Contains("Total_Time_Report") Then
Continue For
End If
all_lines = File.ReadAllLines(Files)
Try
format = findformat(all_lines(1))
starttime = DateTime.ParseExact(all_lines(1).Substring(0, all_lines(1).IndexOf(delimiter)), format, Nothing)
endtime = DateTime.ParseExact(all_lines(all_lines.Length - 1).Substring(0, all_lines(1).IndexOf(delimiter)), format, Nothing)
totaltime = totaltime + Math.Abs(DateDiff(DateInterval.Hour, starttime, endtime)) - 1
output.WriteLine(Files & "," & Math.Abs(DateDiff(DateInterval.Hour, starttime, endtime)) - 1 & ",hours")
Catch ex As Exception
output.WriteLine(Files & ", Unrecognized Date Format")
End Try
Next
output.WriteLine("Total Time," & totaltime & ",hours")
output.Close()
ProgressBar1.Value = ProgressBar1.Maximum
End Sub
Public Function findformat(ByVal input As String) As String
Dim format1 As String = "dd-MM-yyyy HH:mm:ss"
Dim format2 As String = "dd-MM-yyyy hh:mm:ss"
Dim format3 As String = "dd/MM/yyyy HH:mm:ss.fff"
Dim format4 As String = "dd/MM/yyyy hh:mm:ss.fff"
Dim format5 As String = "dd-MM-yyyy HH:mm:ss.fff"
Dim format6 As String = "dd-MM-yyyy hh:mm:ss.fff"
input = input.Substring(0, input.IndexOf("|"))
If (Charactercount(input, "-") = 2) And (Charactercount(input, ":") = 2) And (Charactercount(input, ".") = 1) Then
Return format6
ElseIf (Charactercount(input, "/") = 2) And (Charactercount(input, ":") = 2) And (Charactercount(input, ".") = 1) Then
Return format4
ElseIf (Charactercount(input, "-") = 2) And (Charactercount(input, ":") = 2) Then
Return format2
End If
Return 0
End Function
Public Function Charactercount(ByVal input As String, ByVal find As Char) As Double
Dim counter As Integer = 0
For Each x As Char In input
If x = find Then
counter = counter + 1
End If
Next
Return counter
End Function
End Class