[Edited]
I have a Excel workbook (.xlsx) with two worksheets (Sheet 1 & Sheet 2). Sheet 1 has 7 columns of data (each with about 70k rows) while Sheet 2 only has 5 columns with about 250-500 rows. The first column of each sheet contains a timestamp of when the data was collected in the format (yyyy-mm-dd_hh:mm:ss).
The discrepancy is that the data in Sheet 1 has data points spaced at 8 second intervals. Sheet 2, however, has sporadic data entries. There might be 4 or 5 entries that happen in a burst (say 5 second interval) and then not another entry for another couple of hours.
What I'd like to do is reorganize Sheet 2 so that the timestamps align with Sheet 1. The reason for this is that when I graph the data in sheet 1, the visualization looks appropriate because the data is evenly spaced throughout. However, I can not graph the data in Sheet 2 the same way because the data points occur at sporadic intervals.
I'm comfortable in C# and considering trying to create a program which will read in a csv file of each sheet and do the batch processing... but even there I'm a bit stuck as to what the proper procedure would be. Is there a way this can be handled directly in Excel? Any advice would be welcome.
A bit of background. I tested this on three sheets. First sheet has 100k dates with 8-second intervals. Second sheet, I have 5 columns of 300 data points, with first column containing the dates with sporadic intervals. I've decided against mangling the second sheet so my output is in a third sheet for testing purposes.
Our logic is locating the largest value that's smaller than our target date. This way, we're inside the 8 seconds between this located date and the next one. We then get that value's row from the first sheet, then we use that row as the same row number in our results sheet. We then "transfer" the values from the second sheet to the proper row in the results sheet.
Runtime is negligible on my machine. Hopefully, this runs for you properly as well. Kindly test on a copy of your workbook.
Sub Align()
Dim RefWS As Worksheet, ListWS As Worksheet, ResWS As Worksheet
Dim RngOne As Range, RngTwo As Range
Dim RngVal As Variant, Elem As Variant
Dim LRowOne As Long, LRowTwo As Long, LRowThree As Long
Dim LocRow As Long, RowCt As Long
Dim PopRng As Range, StartRow As Long
With ThisWorkbook
Set RefWS = .Sheets("Sheet1") 'Modify as necessary.
Set ListWS = .Sheets("Sheet2") 'Modify as necessary.
Set ResWS = .Sheets("Sheet3") 'Modify as necessary.
End With
LRowOne = RefWS.Range("A" & Rows.Count).End(xlUp).Row
LRowTwo = ListWS.Range("A" & Rows.Count).End(xlUp).Row
'Make sure to change based on whether you have headers or not.
Set RngOne = RefWS.Range("A1:A" & LRowOne) 'Modify as necessary.
Set RngTwo = ListWS.Range("A1:A" & LRowTwo) 'Modify as necessary.
RngVal = RngTwo.Value
'Change RowCt to 2 if you have headers.
RowCt = 1
For Each Elem In RngVal
LocRow = Application.Match(CDbl(Elem), RngOne, 1)
ResWS.Range("A" & LocRow & ":E" & LocRow).Value = ListWS.Range("A" & RowCt & ":E" & RowCt).Value
RowCt = RowCt + 1
Next Elem
'Autopopulate.
With ResWS
LRowThree = .Range("A" & Rows.Count).End(xlUp).Row
Do
StartRow = .Range("A" & LRowThree).End(xlUp).Row
If StartRow > 1 Then StartRow = StartRow + 1
Set PopRng = .Range("A" & StartRow & ":E" & LRowThree)
.Range("A" & LRowThree & ":E" & LRowThree).Copy
PopRng.PasteSpecial xlPasteValues
LRowThree = StartRow - 1
Loop Until StartRow = 1
End With
Application.CutCopyMode = False
End Sub
It's also important to note that if two values are matched, it's going to get the latest value rather than the closest one. Let me know first what happens to your data after running this.
EDIT: Code updated as per chat.
Related
I have exported CSV files from a Development SQL Server and another from Production.
The table (in the database) has two columns
UserID
DocumentID
both of these should be unique values.
I want to be able to verify that those two combinations (together) match the other environment.
So far I imported both CSV files in separate worksheets in Excel.
After this, I am not sure what I should do to compare these columns?
I did a little google-ing and there are so many different types of answers but not sure how to do it.
Conditional Formatting only works if I select a single column. I need to get the combination of both columns.
A quick and mildly dirty VBA-approach. I assumed your workbook consists of two worksheets, each containing two columns with headers.
Option Explicit
Sub SoftwareIsFun()
Dim wks1 As Worksheet
Dim wks2 As Worksheet
Dim dicObj As Object
Dim lastRow1 As Long
Dim lastRow2 As Long
Dim i As Long
Set dicObj = CreateObject("Scripting.Dictionary")
Set wks1 = ThisWorkbook.Worksheets(1)
Set wks2 = ThisWorkbook.Worksheets(2)
With wks1
lastRow1 = .Cells(.Rows.Count, 1).End(xlUp).Row
For i = 2 To lastRow1
If Not dicObj.Exists(.Range("A" & i).Value) Then
dicObj.Add .Range("A" & i).Value, .Range("B" & i).Value
Else
.Range("C" & i).Value = "UserID already exists"
End If
Next i
End With
With wks2
lastRow2 = .Cells(.Rows.Count, 1).End(xlUp).Row
For i = 2 To lastRow2
If dicObj.Exists(.Range("A" & i).Value) Then
If .Range("B" & i).Value = dicObj.Item(.Range("A" & i).Value) Then
.Rows(i).Interior.Color = vbGreen
Else
.Rows(i).Interior.Color = vbRed
End If
Else
.Rows(i).Interior.Color = vbRed
End If
Next i
End With
End Sub
What you are describing is something I do daily for my job:
Step 1
Create a 3rd column in both worksheets called "Key" where you'll concatenate the values for Column's A & B as follows:=A2&B2.
Now autofill your rows in column C with the previous formula you've written.
Step 2
Remove duplicates found in this column you've created, this will effectively preserve pairs and prevent information loss when removing duplicate values. (Data Tab -> Remove Duplicates -> Select column C as the criteria to remove them).
Step 3
Make a Vlookup in a 4th column in your first worksheet, the function takes 4 parameters: =vlookup(C2, <4th column of the other worksheet (select entire range from row 2 to end)>, 1, 0) and autofill your rows with the formula.
If you aren't yet familiar with vlookup yet I strongly advice you watch a brief tutorial on its usage, it is an essential tool to compare data.
Any value that matches will be displayed, whereas an #N/D error will print for those which don't match between the 2 tables.
I have biomechanics data (i.e. a person walking) that is collected with video recordings and stored as 3 columns of data for each frame number. In a separate data table in the same worksheet, I have data that tells me which excel row number each biomechanic event happens (i.e. the foot strikes the ground or comes up off of the ground). I am trying to take the biomechanics data that is stored in one spreadsheet in rows, and transpose it so that each step is in its own column. However, each biomechanic event (i.e. each step) is a different number of rows.
The biomechanics raw data can be seen here: biomechanics raw data
What I want it to do is copy individual events (steps) to a second worksheet like this: desired biomechanic event output
Instead, the code runs through the loop successfully for each biomechanic event, but places the data in the same place, "A4". This is because I do not know how to use offset within a loop and based on a variable: current biomechanic event output with undesired overlay of data
The code I have currently is here:
Private Sub CommandButton1_Click()
Dim ws As Worksheet: Set ws = ThisWorkbook.Sheets("Sheet1")
Dim ws2 As Worksheet: Set ws2 = ThisWorkbook.Sheets("Sheet2")
Dim LeftStrike As Range, FrameLTD As Range, FrameLTDx As Range
Dim lrL As Long, LastFrame As Long
Dim LeftTD As Variant
Dim LeftTDx As Variant
lrL = ws.Range("H" & ws.Rows.Count).End(xlUp).Row
LastFrame = ws.Range("A" & ws.Rows.Count).End(xlUp).Row
Set LeftStrike = ws.Range("H2:H" & lrL)
For Each FrameLTD In LeftStrike
If InStr(FrameLTD, "Foot Strike") Then
LeftTD = FrameLTD.Offset(0, 2)
'COMMENT: Set LeftTDx = LeftTD + 1
ws.Range("A" & LeftTD, "D" & LastFrame).Copy ws2.Range("A4")
' COMMENT: ws.Range("A" & LeftTDx, "D" & LastFrame).Copy ws2.Range("FrameLTD").Offset(0, 5)
End If
Next FrameLTD
End Sub
What I want the program to do is:
1. Find the string "Foot Strike" and tell me the value 2 columns over and call it LeftTD (it does this).
2. Starting at the row number value of LeftTD, copy cells in columns A through D and past it into a new worksheet starting at A4 (it does this).
3. For the next "Foot Strike" and all remaining, do the same thing in steps 1 and 2, but then copy the cells with an offset(0,5) from the previous copy & pasted event (it does not do this).
4. Do this until column H is empty (it does do this).
My thought was that if I declare the variables as "variant" then I could tell it to count the next LeftTD as LeftTD +1, then tell it to offset the range of the range variable by 5, it would work. But if I un-comment those lines, then I get a type mismatch error. Also, interestingly, if I hit F8 and run through the program step by step, it only copies each biomechanic event (each step) on the 2nd, 5th, 7th, and 9th time through the program.
So my specific question is, how do I use a variable offset within a For loop that is looping through a range variable?
I thought about using .Find and .FindNext, but the information available on the web is really lacking for an example that is close to what I need to do. Indeed, finding information on copy and paste with both variable rows and variable offset columns is also difficult for me to find. Any offered insight would be greatly appreciated. Thank you!!!!
Add a counter to identifying the destination
Dim DestCol As Long
'...
DestCol = 1
For Each FrameLTD In LeftStrike
If InStr(FrameLTD, "Foot Strike") Then
LeftTD = FrameLTD.Offset(0, 2)
ws.Range("A" & LeftTD, "D" & LastFrame).Copy ws2.Cells(4, DestCol)
DestCol = DestCol + 5
End If
Next FrameLTD
Given the lack of elegance with the data
the record sheet continues for many MANY rows, each entry having its own set of identical headings
I was hoping to just extract the data from rows 7, 14 and so on, then populate the data into a simple table to be used on the 'Protocol Summary' form, then sort them all into alphanumeric order based on the data that is in the A column so they all become grouped by 'Event Type'.
Because the potential data that could be under the 'Event Type' heading can vary a lot (generally has the format of [number 1-32/letter/number 1-30] but can also be all letters, with a few thousand possibilities, I thought it might be easier to filter the other lines OUT, given they don't change. I would love to redesign the table, but unfortunately it's not my table so I have to work with what I'm given.
Thanks for your time.
This will loop over your sheet up to the last used row, starting from Row 7 and stepping 7 rows each iteration.
Within each iteration, each cell in the row is written into an array which is then written to another sheet ready for sorting (however you want to do that).
This code is sample and may not work by copy/paste.
I have written this in the Sheet1 code module, so Me refers to ThisWorkbook.Sheets("Sheet1").
I have made this from a blank workbook and did not rename any sheets therefore you will need to make adjustments to any sheet references to match your appropriate sheet names.
The code will only reference columns A, B and C in the TargetRow (I only tested with 3 columns of data as I don't know your working range). I'll reference what to update to extend this after the code block.
Currently the array is put back into Sheet2 starting from cell A2. This is assuming row 1 contains table headers as this will write the data directly into the table format. Naturally if you want to change where the data is written, change the cell it is written to (when writing an array to sheet, you only need to define the top left cell of the range it is written to, Excel works out the rest based on the size and dimensions of the array).
Sub WriteEverySeventhRowToAnotherSheet()
Dim SeventhRowCount As Long
Dim myArray() As Variant
Dim lastrow As Long
Dim TargetCell As Variant
Dim TargetRow As Range
Dim ArrFirstDimension As Long
Dim ArrSecondDimension As Long
lastrow = Me.Range("A" & Me.Rows.Count).End(xlUp).Row
ReDim myArray(1 To lastrow / 7, 1 To 3)
ArrFirstDimension = 1
ArrSecondDimension = 1
'------------------Loop over every 7th row and enter row data to array---------------
For SeventhRowCount = 7 To lastrow Step 7
Set TargetRow = Me.Range("A" & SeventhRowCount & ":C" & SeventhRowCount)
For Each TargetCell In TargetRow
If Not ArrSecondDimension > UBound(myArray) Then
myArray(ArrFirstDimension, ArrSecondDimension) = TargetCell
'Debug.Print TargetCell
ArrSecondDimension = ArrSecondDimension + 1
End If
Next TargetCell
ArrFirstDimension = ArrFirstDimension + 1
ArrSecondDimension = 1
Set TargetRow = Nothing
Next SeventhRowCount
'---------------------Write array to another sheet------------------
Dim Destination As Range
Set Destination = ThisWorkbook.Sheets("Sheet2").Range("A2")
Destination.Resize(UBound(myArray, 1), UBound(myArray, 2)).Value = myArray
End Sub
To extend the number of columns the loop will write to the array, change the following instance of C to the correct column letter (in the below line the range is set from Column A to Column C):
Set TargetRow = Me.Range("A" & SeventhRowCount & ":C" & SeventhRowCount)
Also change the 2nd dimension of the Array to match the number of the Column set above (i.e Column E = 5 and Column L = 13 etc.) - You need to replace the number 3 with the correct number.
ReDim myArray(1 To lastrow / 7, 1 To 3)
I'm trying to come up with a tool to be able to consolidate row data into one line item in Excel. Below is an example:
I want to be able to consolidate record 0604, with the staggered data, into one line item across and remove the duplicates.
Note: this is a limited sample. The real data set can contain more columns (up to 60) with the possibility of 20 duplicate ID that are staggered and needs to be consolidated.
The software that I'm extracting the data from provide this as an output. I'm unable to channel this through a SQL software due to limited access. I know SQL will make this a bit easier, but I was wondering if this can be done in Excel.
Scan up the sheet and compare each cell with the one below and copy from below if blank. Delete row below if it has the same ID
Option Explicit
Sub consolidate()
Const SHEET_NAME = "Sheet1"
Const NO_OF_COLS = 30
Dim wb As Workbook, ws As Worksheet
Dim irow As Long, iLastRow As Long, c As Long, count As Long
Set wb = ThisWorkbook
Set ws = wb.Sheets(SHEET_NAME)
iLastRow = ws.Range("A" & Rows.count).End(xlUp).Row
' scan up sheet
For irow = iLastRow - 1 To 2 Step -1
' if same id below
If ws.Cells(irow + 1, 1) = ws.Cells(irow, 1) Then
' scan across
For c = 1 To NO_OF_COLS
' if blank copy from below
If Len(ws.Cells(irow, c)) = 0 Then
ws.Cells(irow, c) = ws.Cells(irow + 1, c)
End If
Next
ws.Rows(irow + 1).Delete
count = count + 1
End If
Next
MsgBox iLastRow - 1 & " rows scanned" & vbCr & _
count & " rows deleted from " & ws.Name, vbInformation
End Sub
If I understand right you could do this with the concatenate function in excel to combine all of the cells together in one cell.
I believe this is what you were asking?
This is an example of how to use that:
=CONCAT(A1,"-",B1)
And whatever cell this is typed into will have cell "A1-B1" as a value (with the values within those cells instead of the cell names).
I have an issue when trying to use Union(Range, Range). I am trying to copy certain rows from one worksheet and paste them in a new file. My issue is that the Union isn't adding on more rows to the range; it returns the original range. If I flip the order of the parameters, it returns only the .Rows(i + 1) row. My test data has 2 rows that it should copy. The row count at the end is 1. What am I doing wrong?!
Dim lastRow, i As Long
Dim CopyRange As Range
lastRow = ActiveSheet.Rows.count
With Sheets(ActiveSheet.Name)
lastRow = .Range("A" & .Rows.count).End(xlUp).Row
For i = 2 To lastRow
Dim endTime As Date
endTime = DateValue(Trim(.Range("E" & i).Value))
If endTime = Date - 1 Then
If CopyRange Is Nothing Then
Set CopyRange = .Rows(i + 1)
Else
Set CopyRange = Union(CopyRange, .Rows(i + 1))
End If
End If
Next
End With
CopyRange.Copy
Actually CopyRange contains a number of separate ranges (areas). See here for more information.
When you make Union operation with separate rows (I mean that there is another row beetween them) it doesen't actually add Row, but add new Area. So, if you add MsgBox CopyRange.Areas.Count, you will see count of areas (if you will add MsgBox CopyRange.Rows.Count it will get you uncorrect result in case of many areas - it will get count of rows in first area).
As a conclusion, your code works well for me, and should works well for you. You can add CopyRange.Select line before CopyRange.Copy and set breakpoint on this line. You will see that union works well