I have hundreds of csv files with filenames in the following format: yyyymmdd_something.csv, e.g. 20131213_something.csv ... i.e., an underscore separates the date from the rest of the filename.
Each of has the following fields:
Make, Model, metric 1, metric 2, etc.
I would like to:
1) Insert the date from the file name into (preferably first) column, like so:
Date, Make, Model, metric 1, metric 2, etc.
But really it can be any column because once in Excel, it is a simple matter to re-arrange it. The date should be added as mm/dd/yyyy as it is what Excel understands.
2) After inserting the date, I would like to merge all the csv into 1 big csv file.
I'm using a Win7 machine so a dos batch file would probably be the simplest way to run it for me but if I had to, I can install perl or something to do this. Any help would be deeply appreciated. There was a thread that talks about inserting the whole file name but I really need only the date part added.
Hi this can be done by vbscript, the code is below:
Create a VBS file (notepad can do it, just paste following code and save file as .vbs)
You can drop 20-50 files (yyyymmdd_something.csv) as the same time to the icon of this vbs file and it will process as you wish :)
Please create Summary.csv file first and update its full path to pSumaryCSV = "......\Summary.csv"
'USAGE:
'CREATE A VBSCRIPT FILE .VBS WITH THIS CONTENT
'CREATE SUMMARY CSV FILE AND UPDATE ITS FULL PATH IN pSumaryCSV
'DRAG AND DROP YOUR ORIGINAL CSV FILE TO THIS VBS FILE ICON, IT CAN PROCESS MULTIPLE FILE (BUT DON'T PUT TOO MANY AS ONE)
'THIS CODE WILL CREATE A NEW CSV FILE <ORIGINAL FILE NAME>_DATE_ADDED.csv
'AND UPDATE Summary.csv file.
Set objArgs = WScript.Arguments
Set objFso = createobject("scripting.filesystemobject")
dim objOrgFile
dim arrStr ' an array to hold the text content
dim sLine ' holding text to write to new file
'Location of the summary file - Full path. If it is not exist then create it first.
'The summary one should have all column lable since following code will not add label to it.
pSumaryCSV = "......\Summary.csv"
'Open the summary file to append data
set aSummaryFile = objFso.OpenTextFile(pSumaryCSV, 8) '2=Open for writing 8 for appending
'Looping through all dropped file
For t = 0 to objArgs.Count - 1
' Input Path
inPath = objFso.GetFile(wscript.arguments.item(t))
inName = objFso.GetFileName(inPath)
' OutPut Path
outPath = replace(inPath, objFso.GetFileName(inPath), left(inName, InStrRev(objFso.GetFileName(inPath),".") - 1) & "_DATE_ADDED.csv")
' The original file
set objOrgFile = objFso.OpenTextFile(inPath)
'Now Creating the file can overwrite exiting file with same name
set aNewFile = objFso.CreateTextFile(outPath, True)
aNewFile.Close
'Open the new file (...._DATE_ADDED.csv) to appending data
set aNewFile = objFso.OpenTextFile(outPath, 8) '2=Open for writing 8 for appending
'=======================================================================
'Process first line, this firstline will not be added to SummaryCSV File
If Not objOrgFile.AtEndOfStream Then
arrStr = split(objOrgFile.ReadLine,",")
sLine = "Date," 'This will add Date label for
For i=lbound(arrStr) to ubound(arrStr)
sLine = sLine + arrStr(i) + ","
Next
'Writing first line to new file but not the summary one.
aNewFile.WriteLine left(sLine, len(sLine)-1) 'Get rid of that extra comma from the loop
end if
'=======================================================================
' Reading subsequent line and writing it to new file
Do Until objOrgFile.AtEndOfStream
arrStr = split(objOrgFile.ReadLine,",")
'Get the mm/dd/yyyy path from file name yyyymmdd
sLine = ""
sLine = sLine + Mid(inName,5,2) + "/" 'Get mm from file name
sLine = sLine + Mid(inName,7,2) + "/" 'Get dd from file name
sLine = sLine + Mid(inName,1,4) + "/" 'Get yyyy from file name
sLine = Sline + "," 'This will add a column
For i=lbound(arrStr) to ubound(arrStr)
sLine = sLine + arrStr(i) + ","
Next
'Writing data to new file
aNewFile.WriteLine left(sLine, len(sLine)-1) 'Get rid of that extra comma from the loop
'Writing data to summary file
aSummaryFile.WriteLine left(sLine, len(sLine)-1)
Loop
'Closing new file
aNewFile.Close
Next ' This is for next file
'Close Summary File
aSummaryFile.Close
set aSummaryFile=nothing
set aNewFile=nothing
set objFso = nothing
set objArgs = nothing
Related
I am trying to import special characters from a txt file into excel.
I've tried so many things but the characters BREAK in excel.
example of my string:
in txt: Changjíhuízúzìzhìzhou
converts in excel to: ChangjÃhuÃzúzìzhìzhou
so I tried moving values over bit by bit but no luck..
Sub ImportTXTFile()
Dim file As Variant
Dim EXT As String
Dim Direct As String ' directory...
Direct = "C:\FilePath\Here\"
EXT = ".txt"
Dim COL As Long
Dim row As Long
COL = 1
row = 1
file = Dir(Direct)
Do While (file <> "") ' Cycle through files until no more files
If InStr(file, "Data.txt") > 0 Then
'
Open Direct & "Data.txt" For Input As #1
'
While Not EOF(1)
Line Input #1, DataLine ' Read in line
Do While DataLine <> ""
If InStr(DataLine, ",") = 0 Then ' Drop value into excel upto the first ,
Sheets("test").Cells(row, COL).Value = DataLine
DataLine = ""
Else
Sheets("test").Cells(row, COL).Value = Left(DataLine, InStr(DataLine, ",") - 1)
DataLine = Right(DataLine, Len(DataLine) - InStr(DataLine, ",")) ' rebuild array without data upto first ,
End If
COL = COL + 1 ' next column
Loop
COL = 1 ' reset column
row = row + 1 ' write to next row
Wend
'
Close #1 ' Close files straight away
End If
file = Dir
Loop
MsgBox "Data Updated"
End Sub
So I want to cry because all this converting of UTF-8 to ASCII can be avoid simply by:
opening the txt file in Notepad++
going to the encoding tab
clicking convert to ASCII
ran my original code.
BLAM
everything is perfect.
Thank you danieltakeshi for all your help!
Using the first link i gave you, here is a test code, i tested with success. Using the charset: CdoISO_8859_1
Dim objStream As Object
Dim strData As String
Set objStream = CreateObject("ADODB.Stream")
objStream.Charset = "iso-8859-1"
objStream.Open
objStream.LoadFromFile ("C:\Users\user_name\Desktop\test.txt")
strData = objStream.ReadText()
Debug.Print strData & " Compare to: Changjíhuízúzìzhìzhou"
The output was:
EDIT:
Check the encoding type of your .txt file and import to Excel with the same encoding charset, for example, i changed the test.txt to UTF-8 and imported successfully with the .Charset as "utf-8"
You can Save As your .txt file and choose the encoding.
Currently i using VBA code to export range data to a CSV file:
Sub Fct_Export_CSV_Migration() Dim Value As String Dim size As Integer
Value = ThisWorkbook.Path & "\Export_Migration" & Sheets(1).range("B20").Value & ".csv" chemincsv = Value
Worksheets("Correspondance Nv Arborescence").Select Dim Plage As Object, oL As Object, oC As Object, Tmp As String, Sep$ Sep = ";" size = Worksheets("Correspondance Nv Arborescence").range("B" & Rows.Count).End(xlUp).Row Set Plage = ActiveSheet.range("A1:B" & size)
Open chemincsv For Output As #1 For Each oL In Plage.Rows
Tmp = ""
For Each oC In oL.Cells
Tmp = Tmp & CStr(oC.Text) & Sep
Next
'take one less than length of the string number of characters from left, that would eliminate the trailing semicolon
Tmp = Left(Tmp, Len(Tmp) - 1)
Print #1, Tmp Next Close
MsgBox "OK! Export to " & Value End Sub
Now, i would like to export CSV encoded with "Unicode". I think i need to use VBA function like SaveAs( xlUnicodeText ) but how to use that ?
Thx
Unicode CSVs are not one of the file formats supported by Excel, out of the box. This means we cannot use the SaveAs method. The good news we can work around this restriction, using VBA.
My approach uses the file system object. This incredibly handy object is great for interacting with the file system. Before you can use it you will need to add a reference:
From the VBA IDE click Tools.
Click References...
Select Windows Script Host Object Model from the list.
Press OK.
The code:
' Saves the active sheet as a Unicode CSV.
Sub SaveAsUnicodeCSV()
Dim fso As FileSystemObject ' Provides access to the file system.
Dim ts As TextStream ' Writes to your text file.
Dim r As Range ' Used to loop over all used rows.
Dim c As Range ' Used to loop over all used columns.
' Use the file system object to write to the file system.
' WARNING: This code will overwrite any existing file with the same name.
Set fso = New FileSystemObject
Set ts = fso.CreateTextFile("!!YOUR FILE PATH HERE.CSV!!", True, True)
' Read each used row.
For Each r In ActiveSheet.UsedRange.Rows
' Read each used column.
For Each c In r.Cells
' Write content to file.
ts.Write c.Value
If c.Column < r.Columns.Count Then ts.Write ","
Next
' Add a line break, between rows.
If r.Row < ActiveSheet.UsedRange.Count Then ts.Write vbCrLf
Next
' Close the file.
ts.Close
' Release object variables before they leave scope, to reclaim memory and avoid leaks.
Set ts = Nothing
Set fso = Nothing
End Sub
This code loops over each used row in the active worksheet. Within each row, it loops over every column in use. The contents of each cell is appended to your text file. At the end of each row, a line break is added.
To use; simply replace !!YOUR FILE PATH HERE.CSV!! with your file name.
I need to export data from Excel to a comma delimited file. I am using a button that runs a macro that creates the text file in a location I specify and exports values.
But, I want to copy this Excel with the same macro, and add different values. When I run the macro in the copied Excel file I want the same text delimited file to add these values under the previous set of values, not overwrite the values from the first Excel file.
How is this possible? I want to use the same text file each time.
Append using following routine:
Sub writeCSV(ByVal thisRange As Range, ByVal filePath As String, _
Optional ByVal fileAppend As Boolean = False)
Dim cLoop As Long, rLoop As Long
Dim ff As Long, strRow As String
ff = FreeFile
If fileAppend Then
Open filePath For Append As #ff
Else
Open filePath For Output As #ff
End If
For rLoop = 1 To thisRange.Rows.Count
strRow = ""
For cLoop = 1 To thisRange.Columns.Count
If cLoop > 1 Then strRow = strRow & ","
strRow = strRow & thisRange.Cells(rLoop, cLoop).Value
Next 'cLoop
Print #ff, strRow
Next 'rLoop
Close #ff
End Sub
Example usage
writeCSV sheet1, "c:\test.txt", true
It is easy to do it by using a StreamWriter. The AppendText function of the FileInfo class returns one that is configured for appending text to an existing file.
I write my example as pseudo code, as I don't know how you are retrieving the information from Excel. The File handling part, however, is complete:
Dim file As New FileInfo("C:\myOuputFile.csv")
Using writer = file.AppendText()
While isRowAvailable
writer.WriteLine("Write next row")
End While
End Using
The Using statement automatically closes the file at the end.
I've found an answer to import lines of data from numerous text files into an Excel sheet (https://stackoverflow.com/a/4941605/1892030 answered by Chris Neilsen). However I would like to also do the following:
There is garbage data before and after the useful data I want to import. The lines of data I want to import all start with an asterix (*).
The data is comma delimited and must be parsed that way when imported into Excel. This I could change by editing the parse code in the above answer.
At the end of each line that is imported, I want to add an additional item of data which is the name of the text file where the data was imported from (name of file only, without file extension).
The answer from Chris refered to above works real well so I would like to edit the code to allow for my additional requirements under points 1 and 3 above - but don't know how. For completeness I copy the code from the earlier answer below. Many thanks.
Sub ReadFilesIntoActiveSheet()
Dim fso As FileSystemObject
Dim folder As folder
Dim file As file
Dim FileText As TextStream
Dim TextLine As String
Dim Items() As String
Dim i As Long
Dim cl As Range
' Get a FileSystem object
Set fso = New FileSystemObject
' get the directory you want
Set folder = fso.GetFolder("C:\#test")
' set the starting point to write the data to
Set cl = ActiveSheet.Cells(1, 1)
' Loop thru all files in the folder
For Each file In folder.Files
' Open the file
Set FileText = file.OpenAsTextStream(ForReading)
' Read the file one line at a time
Do While Not FileText.AtEndOfStream
TextLine = FileText.ReadLine
' Parse the line into comma delimited pieces
Items = Split(TextLine, ",")
' Put data on one row in active sheet
For i = 0 To UBound(Items)
cl.Offset(0, i).Value = Items(i)
Next
' Move to next row
Set cl = cl.Offset(1, 0)
Loop
' Clean up
FileText.Close
Next file
Set FileText = Nothing
Set file = Nothing
Set folder = Nothing
Set fso = Nothing
End Sub
I haven't done it all for you (I expect the file name will need tidying up to fit the format you want) but drop this code in and it will get you started...
' Read the file one line at a time
Do While Not FileText.AtEndOfStream
TextLine = FileText.ReadLine
' Process lines which don't begin with Asterisk (*)
If Left(TextLine,1)<>"*" Then
' This crudely appends the filename as if it were a column in the source file
TextLine = TextLine + "," + file.Name
' Parse the line into comma delimited pieces
Items = Split(TextLine, ",")
' Put data on one row in active sheet
For i = 0 To UBound(Items)
cl.Offset(0, i).Value = Items(i)
Next
' Move to next row
Set cl = cl.Offset(1, 0)
End If
Loop
I have one folder with multiple text files that I add one text file to every day. All text files are in the same format and are pipe delimited.
Is it possible to create code for excel that will automatically import the data from the multiple text files into one worksheet?
I found some code that would import all the text files from the folder, but only if I changed it all to comma delimited first. Also, I could not get it to update if I added files to the folder.
Any help would be greatly appreciated!
A good way to handle files in general is the 'FileSystemObject'. To make this available in VBA you need to add a reference to it:
(select the Tools\References menu. In the References dialog, select 'Microsoft Scripting Runtime')
The following code example will read all files in a folder, reads their content one line at a time, split each line into | separated bits, and writes theses bits to the active sheet starting at cell A1, one row per line.
Sub ReadFilesIntoActiveSheet()
Dim fso As FileSystemObject
Dim folder As folder
Dim file As file
Dim FileText As TextStream
Dim TextLine As String
Dim Items() As String
Dim i As Long
Dim cl As Range
' Get a FileSystem object
Set fso = New FileSystemObject
' get the directory you want
Set folder = fso.GetFolder("D:\YourDirectory\")
' set the starting point to write the data to
Set cl = ActiveSheet.Cells(1, 1)
' Loop thru all files in the folder
For Each file In folder.Files
' Open the file
Set FileText = file.OpenAsTextStream(ForReading)
' Read the file one line at a time
Do While Not FileText.AtEndOfStream
TextLine = FileText.ReadLine
' Parse the line into | delimited pieces
Items = Split(TextLine, "|")
' Put data on one row in active sheet
For i = 0 To UBound(Items)
cl.Offset(0, i).Value = Items(i)
Next
' Move to next row
Set cl = cl.Offset(1, 0)
Loop
' Clean up
FileText.Close
Next file
Set FileText = Nothing
Set file = Nothing
Set folder = Nothing
Set fso = Nothing
End Sub
the sub is deliberately simplified to remain clear (i hope) and will need work to be made robust (eg add error handling)
Sounds like it'd be easiser to run a script to cycle thru all the files in the directory, create a new file composed of all the files' contents as new lines, and save that as a csv. something like:
import os
basedir='c://your_root_dir'
dest_csv="<path to wherever you want to save final csv>.csv"
dest_list=[]
for root, subs, files in os.walk(basedir):
for f in files:
thisfile=open(basedir+f)
contents=thisfile.readlines()
dest_list.append(contents)
#all that would create a list containing the contents of all the files in the directory
#now we'll write it as a csv
f_csv=open(dest_csv,'w')
for i in range(len(dest_list)):
f_csv.write(dest_list[i])
f_csv.close()
You could save a script like that somewhere and run it each day, then open the resulting csv in excel. This assumes that you want to get the data from each file in a particular directory, and that all the files you need are in one directory.
You can use Schema.ini ( http://msdn.microsoft.com/en-us/library/ms709353(VS.85).aspx ), Jet driver and Union query, with any luck.