Actually the below program is for Dispersal Algorithm called Rabin-IDA; this algorithm divided the data into N pieces and then recombine it from M pieces (such that M<N).
Thus, the below program needs command line arguments,which entering by Project properties/Debugging.
this argument is file name, where the program performing spitted the file into N files, and then recombine it from M divided files, and put it on another file which should also passing its name as argument .
Now my question is, How can i make this program enter the file name by keyboard??(i mean enter the files name by user from screen not as command line arguments)
the below code is just the main function of program, and the whole of it in this link (http://www.juancamilocorena.com/home/projects) Information Dispersal Algorithms Rabin-IDA.
#include "include.h"
void __cdecl _tmain(int argc, TCHAR *argv[])
{
DWORD ini=GetTickCount();
try
{
if( argc == 3 ) //recombine
{
RabinIDA rabin=RabinIDA(17,10);
long long size=GetFileSize(argv[1]);
int f[]={0,2,3,5,6,8,9,11,14,15};
rabin.recombine(argv[1],
f,
argv[2],
size);
}
else if(argc == 2)
{
RabinIDA rabin=RabinIDA(17,10);
rabin.split(argv[1]);
}
else
{
printf("Error. To split a file pass a parameter with the file to be splitted\n");
printf("To recombine the file give the name of the original file and the output file\n");
printf("The name of the file is used to get the size of the original file only, in a production\n");
printf("environment the length of the original file and the id of the share must be stored along with the share");
return;
}
printf("%d\n",GetTickCount()-ini);
}
catch (int)
{
PrintLastError(_T("MAIN CATCH"));
}
}
If you want to get the file name from the console, you can do this:
cout << "Enter file name: ";
string filename;
getline(cin, filename);
Related
Visual Studio 2013 C++. what I am trying to do is, I will ask for a string and extract each letter from that string. (I know it is already char) but I dont know how to extract each letter, and find out number of chars entered. I do not know how to stop the command line window. getchar() only gets one char but the user may enter several chars continuously, like a string. Any idea? Thanks.
int main(int argc, char *argv[]) {
string firstarg;
if (argc < 1)
{
//Check see if str passed
cout << "No string entered....";
}
else
{
firstarg = argv[1];
cout << firstarg;
}
}
I have a very simple task to do, but somehow I am still stuck.
I have one BIG data file ("File_initial.dat"), which should be read by all nodes on the cluster (using MPI), each node will perform some manipulation on part of this BIG file (File_size / number_of_nodes) and finally each node will write its result to one shared BIG file ("File_final.dat"). The number of elements of files remain the same.
By googling I understood, that it is much better to write data file as a binary file (I have only decimal numbers in this file) and not as *.txt" file. Since no human will read this file, but only computers.
I tried to implement myself (but using formatted in/output and NOT binary file) this, but I get incorrect behavior.
My code so far follows:
#include <fstream>
#define NNN 30
int main(int argc, char **argv)
{
ifstream fin;
// setting MPI environment
int rank, nprocs;
MPI_File file;
MPI_Init(&argc, &argv);
MPI_Comm_size(MPI_COMM_WORLD, &nprocs);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
// reading the initial file
fin.open("initial.txt");
for (int i=0;i<NNN;i++)
{
fin >> res[i];
cout << res[i] << endl; // to see, what I have in the file
}
fin.close();
// starting position in the "res" array as a function of "rank" of process
int Pstart = (NNN / nprocs) * rank ;
// specifying Offset for writing to file
MPI_Offset offset = sizeof(double)*rank;
MPI_File file;
MPI_Status status;
// opening one shared file
MPI_File_open(MPI_COMM_WORLD, "final.txt", MPI_MODE_CREATE|MPI_MODE_WRONLY,
MPI_INFO_NULL, &file);
// setting local for each node array
double * localArray;
localArray = new double [NNN/nprocs];
// Performing some basic manipulation (squaring each element of array)
for (int i=0;i<(NNN / nprocs);i++)
{
localArray[i] = res[Pstart+i]*res[Pstart+i];
}
// Writing the result of each local array to the shared final file:
MPI_File_seek(file, offset, MPI_SEEK_SET);
MPI_File_write(file, localArray, sizeof(double), MPI_DOUBLE, &status);
MPI_File_close(&file);
MPI_Finalize();
return 0;
}
I understand, that I do something wrong, while trying to write double as a text file.
How one should change the code in order to be able to save
as .txt file (format output)
as .dat file (binary file)
Your binary file output is almost right; but your calculations for your offset within the file and the amount of data to write is incorrect. You want your offset to be
MPI_Offset offset = sizeof(double)*Pstart;
not
MPI_Offset offset = sizeof(double)*rank;
otherwise you'll have each rank overwriting each others data as (say) rank 3 out of nprocs=5 starts writing at double number 3 in the file, not (30/5)*3 = 18.
Also, you want each rank to write NNN/nprocs doubles, not sizeof(double) doubles, meaning you want
MPI_File_write(file, localArray, NNN/nprocs, MPI_DOUBLE, &status);
How to write as a text file is a much bigger issue; you have to convert the data into string internally and then output those strings, making sure you know how many characters each line requires by careful formatting. That is described in this answer on this site.
It's only every once in awhile that I have to write in c++. My problem is how do you find a character string within a string? Then save that line, and then look for a set of numbers in that line. For example I have a text file that looks like this.
Address Length Name
87623498 2 dog
12345678 4 cat
98737289 1 bird
I want to search for "cat" and then store the numbers associated with it (12345678) and (4) to different variable names. Here is some code I have written, but it's not close to being correct. I have an .exe that calls this DLL. Any help is appreciated!
`
#include
#include
#include
#include
#include
char* File_Path; //Path is sent by another program
char* Name; //value is being sent by another program
unsigned long Address;
int Length;
int found = 0;
{
using namespace std;
std::string Line;
std::ifstream infile;
infile.open (File_Path);
if (infile.is_open())
{
std::ofstream outfile;
outfile.open ("C:\\debug\\test.txt")
while (infile.good() && found == 0)
{
std::getline(infile,Line);
//if (Name is within Line){
/*I want to say IF char* Name = a part of the string line
ie. Name = cat and the string Line is (12345678 4 cat)
Then store the number (12345678 = Address), and
(4 = Length) I hope that makes sense :/
*/
outfile << Line; //output the stored line for debug
outfile << address; //output the stored address for debug
outfile << length; //output the stored length for debug
found = 1;
}
outfile.close();
infile.close();
}
return 0;
}
`
Please help! Thank you!
Wikipedia has an excellent description of the ZIP file format, but the "central directory" structure is confusing to me. Specifically this:
This ordering allows a ZIP file to be created in one pass, but it is usually decompressed by first reading the central directory at the end.
The problem is that even the trailing header for the central directory is variable length. How then, can someone get the start of the central directory to parse?
(Oh, and I did spend some time looking at APPNOTE.TXT in vain before coming here and asking :P)
My condolences, reading the wikipedia description gives me the very strong impression that you need to do a fair amount of guess + check work:
Hunt backwards from the end for the 0x06054b50 end-of-directory tag, look forward 16 bytes to find the offset for the start-of-directory tag 0x02014b50, and hope that is it. You could do some sanity checks like looking for the comment length and comment string tags after the end-of-directory tag, but it sure feels like Zip decoders work because people don't put funny characters into their zip comments, filenames, and so forth. Based entirely on the wikipedia page, anyhow.
I was implementing zip archive support some time ago, and I search last few kilobytes for a end of central directory signature (4 bytes). That works pretty good, until somebody will put 50kb text into comment (which is unlikely to happen. To be absolutely sure, you can search last 64kb + few bytes, since comment size is 16 bit).
After that, I look up for zip64 end of central dir locator, that's easier since it has fixed structure.
Here is a solution I have just had to roll out incase anybody needs this. This involves grabbing the central directory.
In my case I did not want any of the compression features that are offered in any of the zip solutions. I just wanted to know about the contents. The following code will return a ZipArchive of a listing of every entry in the zip.
It also uses a minimum amount of file access and memory allocation.
TinyZip.cpp
#include "TinyZip.h"
#include <cstdio>
namespace TinyZip
{
#define VALID_ZIP_SIGNATURE 0x04034b50
#define CENTRAL_DIRECTORY_EOCD 0x06054b50 //signature
#define CENTRAL_DIRECTORY_ENTRY_SIGNATURE 0x02014b50
#define PTR_OFFS(type, mem, offs) *((type*)(mem + offs)) //SHOULD BE OK
typedef struct {
unsigned int signature : 32;
unsigned int number_of_disk : 16;
unsigned int disk_where_cd_starts : 16;
unsigned int number_of_cd_records : 16;
unsigned int total_number_of_cd_records : 16;
unsigned int size_of_cd : 32;
unsigned int offset_of_start : 32;
unsigned int comment_length : 16;
} ZipEOCD;
ZipArchive* ZipArchive::GetArchive(const char *filepath)
{
FILE *pFile = nullptr;
#ifdef WIN32
errno_t err;
if ((err = fopen_s(&pFile, filepath, "rb")) == 0)
#else
if ((pFile = fopen(filepath, "rb")) == NULL)
#endif
{
int fileSignature = 0;
//Seek to start and read zip header
fread(&fileSignature, sizeof(int), 1, pFile);
if (fileSignature != VALID_ZIP_SIGNATURE) return false;
//Grab the file size
long fileSize = 0;
long currPos = 0;
fseek(pFile, 0L, SEEK_END);
fileSize = ftell(pFile);
fseek(pFile, 0L, SEEK_SET);
//Step back the size of the ZipEOCD
//If it doesn't have any comments, should get an instant signature match
currPos = fileSize;
int signature = 0;
while (currPos > 0)
{
fseek(pFile, currPos, SEEK_SET);
fread(&signature, sizeof(int), 1, pFile);
if (signature == CENTRAL_DIRECTORY_EOCD)
{
break;
}
currPos -= sizeof(char); //step back one byte
}
if (currPos != 0)
{
ZipEOCD zipOECD;
fseek(pFile, currPos, SEEK_SET);
fread(&zipOECD, sizeof(ZipEOCD), 1, pFile);
long memBlockSize = fileSize - zipOECD.offset_of_start;
//Allocate zip archive of size
ZipArchive *pArchive = new ZipArchive(memBlockSize);
//Read in the whole central directory (also includes the ZipEOCD...)
fseek(pFile, zipOECD.offset_of_start, SEEK_SET);
fread((void*)pArchive->m_MemBlock, memBlockSize - 10, 1, pFile);
long currMemBlockPos = 0;
long currNullTerminatorPos = -1;
while (currMemBlockPos < memBlockSize)
{
int sig = PTR_OFFS(int, pArchive->m_MemBlock, currMemBlockPos);
if (sig != CENTRAL_DIRECTORY_ENTRY_SIGNATURE)
{
if (sig == CENTRAL_DIRECTORY_EOCD) return pArchive;
return nullptr; //something went wrong
}
if (currNullTerminatorPos > 0)
{
pArchive->m_MemBlock[currNullTerminatorPos] = '\0';
currNullTerminatorPos = -1;
}
const long offsToFilenameLen = 28;
const long offsToFieldLen = 30;
const long offsetToFilename = 46;
int filenameLength = PTR_OFFS(int, pArchive->m_MemBlock, currMemBlockPos + offsToFilenameLen);
int extraFieldLen = PTR_OFFS(int, pArchive->m_MemBlock, currMemBlockPos + offsToFieldLen);
const char *pFilepath = &pArchive->m_MemBlock[currMemBlockPos + offsetToFilename];
currNullTerminatorPos = (currMemBlockPos + offsetToFilename) + filenameLength;
pArchive->m_Entries.push_back(pFilepath);
currMemBlockPos += (offsetToFilename + filenameLength + extraFieldLen);
}
return pArchive;
}
}
return nullptr;
}
ZipArchive::ZipArchive(long size)
{
m_MemBlock = new char[size];
}
ZipArchive::~ZipArchive()
{
delete[] m_MemBlock;
}
const std::vector<const char*> &ZipArchive::GetEntries()
{
return m_Entries;
}
}
TinyZip.h
#ifndef __TinyZip__
#define __TinyZip__
#include <vector>
#include <string>
namespace TinyZip
{
class ZipArchive
{
public:
ZipArchive(long memBlockSize);
~ZipArchive();
static ZipArchive* GetArchive(const char *filepath);
const std::vector<const char*> &GetEntries();
private:
std::vector<const char*> m_Entries;
char *m_MemBlock;
};
}
#endif
Usage:
TinyZip::ZipArchive *pArchive = TinyZip::ZipArchive::GetArchive("Scripts_unencrypt.pak");
if (pArchive != nullptr)
{
const std::vector<const char*> entries = pArchive->GetEntries();
for (auto entry : entries)
{
//do stuff
}
}
In case someone out there is still struggling with this problem - have a look at the repository I hosted on GitHub containing my project that could answer your questions.
Zip file reader
Basically what it does is download the central directory part of the .zip file which resides in the end of the file.
Then it will read out every file and folder name with it's path from the bytes and print it out to console.
I have made comments about the more complicated steps in my source code.
The program can work only till about 4GB .zip files. After that you will have to do some changes to the VM size and maybe more.
Enjoy :)
I recently encountered a similar use-case and figured I would share my solution for posterity since this post helped send me in the right direction.
Using the Zip file central directory offsets detailed on Wikipedia here, we can take the following approach to parse the central directory and retrieve a list of the contained files:
STEPS:
Find the end of the central directory record (EOCDR) by scanning the zip file in binary format for the EOCDR signature (0x06054b50), beginning at the end of the file (i.e. read the file in reverse using std::ios::ate if using a ifstream)
Use the offset located in the EOCDR (16 bytes from the EOCDR) to position the stream reader at the beginning of the central directory
Use the offset (46 bytes from the CD start) to position the stream reader at the file name and track its position start point
Scan until either another central directory header is found (0x02014b50) or the EOCDR is found, and track the position
Reset the reader to the start of the file name and read until the end
Position the reader over the next header, or terminate if the EOCDR is found
The key point here is that the EOCDR is uniquely identified by a signature (0x06054b50) that occurs only one time. Using the 16 byte offset, we can position ourselves to the first occurrence of the central directory header (0x02014b50). Each record will have the same 0x02014b50 header signature, so you just need to loop through occurrences of the header signatures until you hit the EOCDR ending signature (0x06054b50) again.
SUMMARY:
If you want to see a working example of the above steps, you can check out my minimal implementation (ZipReader) on GitHub here. The implementation can be used like this:
ZipReader zr;
if (zr.SetInput("blah.zip") == ZipReaderStatus::S_FAIL)
std::cout << "set input error" << std::endl;
std::vector<std::string> entries;
if (zr.GetEntries(entries) == ZipReaderStatus::S_FAIL)
std::cout << "get entries error" << std::endl;
for (auto entry : entries)
std::cout << entry << std::endl;
When viewing (or editing) a .gz file, vim knows to locate gunzip and display the file properly.
In such cases, getfsize(expand("%")) would be the size of the gzipped file.
Is there a way to get the size of the expanded file?
[EDIT]
Another way to solve this might be getting the size of current buffer, but there seems to be no such function in vim. Am I missing something?
There's no easy way to get the uncompressed size of a gzipped file, short of uncompressing it and using the getfsize() function. That might not be what you want. I took at a look at RFC 1952 - GZIP File Format Specification, and the only thing that might be useful is the ISIZE field, which contains "...the size of the original (uncompressed) input data modulo 2^32".
EDIT:
I don't know if this helps, but here's some proof-of-concept C code I threw together that retrieves the value of the ISIZE field in a gzip'd file. It works for me using Linux and gcc, but your mileage may vary. If you compile the code, and then pass in a gzip'd filename as a parameter, it will tell you the uncompressed size of the original file.
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
int main(int argc, char *argv[])
{
FILE *fp = NULL;
int i=0;
if ( argc != 2 ) {
fprintf(stderr, "Must specify file to process.\n" );
return -1;
}
// Open the file for reading
if (( fp = fopen( argv[1], "r" )) == NULL ) {
fprintf( stderr, "Unable to open %s for reading: %s\n", argv[1], strerror(errno));
return -1;
}
// Look at the first two bytes and make sure it's a gzip file
int c1 = fgetc(fp);
int c2 = fgetc(fp);
if ( c1 != 0x1f || c2 != 0x8b ) {
fprintf( stderr, "File is not a gzipped file.\n" );
return -1;
}
// Seek to four bytes from the end of the file
fseek(fp, -4L, SEEK_END);
// Array containing the last four bytes
unsigned char read[4];
for (i=0; i<4; ++i ) {
int charRead = 0;
if ((charRead = fgetc(fp)) == EOF ) {
// This shouldn't happen
fprintf( stderr, "Read end-of-file" );
exit(1);
}
else
read[i] = (unsigned char)charRead;
}
// Copy the last four bytes into an int. This could also be done
// using a union.
int intval = 0;
memcpy( &intval, &read, 4 );
printf( "The uncompressed filesize was %d bytes (0x%02x hex)\n", intval, intval );
fclose(fp);
return 0;
}
This appears to work for getting the byte count of a buffer
(line2byte(line("$")+1)-1)
If you're on Unix/linux, try
:%!wc -c
That's in bytes. (It works on windows, if you have e.g. cygwin installed.) Then hit u to get your content back.
HTH
From within vim editor, try this:
<Esc>:!wc -c my_zip_file.gz
That will display you the number of bytes the file is having.