I have a encoded string which was encoded using below command on linux machine,
cat <file-name> | gzip | base64 -w0
And I am able to decode and string using below method,
echo '<encoded-string> | base64 -di | zcat
Is there any way to decode the same string on windows machine.
The method:
echo VERY_LARGE_STRING | gzip | base64 -w0
.. is limited by linux setting :
getconf ARG_MAX
Used the below and saved my day. Many thanks to Karthik1234
cat <file-name> | gzip | base64 -w0
Here is how we unzip in C# the message compressed in linux
Note: You may need to remove the last byte.
static byte[] Decompress(byte[] gzip)
{
// Create a GZIP stream with decompression mode.
// ... Then create a buffer and write into while reading from the GZIP stream.
using (GZipStream stream = new GZipStream(new MemoryStream(gzip),
CompressionMode.Decompress))
{
const int size = 4096;
byte[] buffer = new byte[size];
using (MemoryStream memory = new MemoryStream())
{
int count = 0;
do
{
count = stream.Read(buffer, 0, size);
if (count > 0)
{
memory.Write(buffer, 0, count);
}
}
while (count > 0);
return memory.ToArray();
}
}
}
Related
I'm trying to upload a file size of 230GB into azure block blob with the following code
private void uploadFile(FileObject srcFile, FileObject destFile) throws Exception {
try {
BlobClient destBlobClient = blobContainerClient.getBlobClient("destFilename");
long blockSize = 4 * 1024 * 1024 // 4 MB
ParallelTransferOptions opts = new ParallelTransferOptions()
.setBlockSizeLong(blockSize)
.setMaxConcurrency(5);
BlobRequestConditions requestConditions = new BlobRequestConditions();
try (BlobOutputStream bos = destBlobClient.getBlockBlobClient().getBlobOutputStream(
opts, null, null, null, requestConditions);
InputStream is = srcFile.getContent().getInputStream()) {
byte[] buffer = new byte[(int) blockSize];
int i = 0;
for (int len; (len = is.read(buffer)) != -1; ) {
bos.write(buffer, 0, len);
}
}
}
finally {
destFile.close();
srcFile.close();
}
}
Since,I am explicitly setting block size 4MB for each write operation I'm in a assumption that each write block is considered as single block in azure. But which is not the case.
For the above example 230GB file the write operation was executed 58880 times and the file got uploaded successfully.
Can someone please explain me more about how blocks are splits internally in azure and let help me to understand better.
Thanks in advance
How do I encode a String in Ceylon as UTF-8 bytes?
value string = "my_string";
[Byte*] bytes = string.______;
Use ceylon.buffer.charset.
import ceylon.buffer.charset {
utf8
}
shared void run() {
value string = "my_string";
List<Byte> bytes = utf8.encode(string);
Byte[] bytesSequence = bytes.sequence(); // in case a List isn’t enough
}
Try it!
I am running the Portable Device API to automatically get Photos from a connected Smart Phone. I have it all transferring correctly. The code that i use is that Standard DownloadFile() routine:
public PortableDownloadInfo DownloadFile(PortableDeviceFile file, string saveToPath)
{
IPortableDeviceContent content;
_device.Content(out content);
IPortableDeviceResources resources;
content.Transfer(out resources);
PortableDeviceApiLib.IStream wpdStream;
uint optimalTransferSize = 0;
var property = new _tagpropertykey
{
fmtid = new Guid(0xE81E79BE, 0x34F0, 0x41BF, 0xB5, 0x3F, 0xF1, 0xA0, 0x6A, 0xE8, 0x78, 0x42),
pid = 0
};
resources.GetStream(file.Id, ref property, 0, ref optimalTransferSize, out wpdStream);
System.Runtime.InteropServices.ComTypes.IStream sourceStream =
// ReSharper disable once SuspiciousTypeConversion.Global
(System.Runtime.InteropServices.ComTypes.IStream)wpdStream;
var filename = Path.GetFileName(file.Name);
if (string.IsNullOrEmpty(filename))
return null;
FileStream targetStream = new FileStream(Path.Combine(saveToPath, filename),
FileMode.Create, FileAccess.Write);
try
{
unsafe
{
var buffer = new byte[1024];
int bytesRead;
do
{
sourceStream.Read(buffer, 1024, new IntPtr(&bytesRead));
targetStream.Write(buffer, 0, 1024);
} while (bytesRead > 0);
targetStream.Close();
}
}
finally
{
Marshal.ReleaseComObject(sourceStream);
Marshal.ReleaseComObject(wpdStream);
}
return pdi;
}
}
There are two problems with this standard code:
1) - when the images are saves to the windows machine, there is no EXIF information. this information is what i need. how do i preserve it?
2) the saved files are very bloated. for example, the source jpeg is 1,045,807 bytes, whilst the downloaded file is 3,942,840 bytes!. it is similar to all of the other files. I would of thought that the some inside the unsafe{} section would output it byte for byte? Is there a better way to transfer the data? (a safe way?)
Sorry about this. it works fine.. it is something else that is causing these issues
I wrote a dll for an application which wil upload some information from the application via ftp to a server. The upload works fine for most of the files I upload.
There is one file which will upload completely but after the upload it is directly removed from the server (this happens not alway, sometimes the file exists on the server after upload). The file is arround 400 kb, all the other files are smaller.
date and type are two CStrings. Data contains the file content and type the first part of the filename.
CInternetSession session(_T("whtsnXt_dll"));
CFtpConnection* pServer = NULL;
CInternetFile* pFile = NULL;
LPCTSTR pszServerName = _T("servername");
CString fileName = type + L".txt";
int curPos = 0;
CString postData = data;
try
{
CString strServerName;
INTERNET_PORT nPort = 21;
pServer = session.GetFtpConnection(pszServerName, _T("username"), _T("password"), nPort, TRUE);
if (pServer->SetCurrentDirectory(L"goes") == 0) {
// MessageBox::Show("De map bestaat niet", "whtsnXt error", MessageBoxButtons::OK, MessageBoxIcon::Error);
}
pFile = pServer->OpenFile((LPCTSTR)fileName, GENERIC_WRITE);
pFile->WriteString(postData);
pFile->Close();
pServer->Close();
delete pFile;
delete pServer;
}
catch (CInternetException* pEx)
{
//catch errors from WinInet
TCHAR pszError[64];
pEx->GetErrorMessage(pszError, 64);
MessageBox::Show(gcnew String(pszError), "whtsnXt error", MessageBoxButtons::OK, MessageBoxIcon::Error);
}
session.Close();
Does someone know a way to upload that file withoud directly removing?
Try to upload the file in smaller pieces:
int i;
for (i = 0; i < postData.Getlength(); i += 1024)
{
pFile->WriteString(postData.Mid(i, min(1024, postData.Getlength() - i));
}
Just to be sure: data is actually a multi-byte or unicode string and holds no binary data? WriteString will only write till it finds a '\0' character. To upload binary data, use pFile->Write.
I have a website in asp.net 2.0, As I need to use CCNOW payment integration to make a payment but for this I'll have to send request to CCNOW in MD5 format but I can't able to generate my values to CCNOW MD5 format. So, could you please any one have a script/function that will convert given string into MD5?
MD5 isn't a "format," is a hashing algorithm. Use the MD5 class. Assuming you're using C#, it would look something like this:
static string getMd5Hash(string input)
{
// Create a new instance of the MD5CryptoServiceProvider object.
MD5 md5Hasher = MD5.Create();
// Convert the input string to a byte array and compute the hash.
byte[] data = md5Hasher.ComputeHash(Encoding.Default.GetBytes(input));
// Create a new Stringbuilder to collect the bytes
// and create a string.
StringBuilder sBuilder = new StringBuilder();
// Loop through each byte of the hashed data
// and format each one as a hexadecimal string.
for (int i = 0; i < data.Length; i++)
{
sBuilder.Append(data[i].ToString("x2"));
}
// Return the hexadecimal string.
return sBuilder.ToString();
}
public static string GetMD5(string value) {
MD5 md5 = MD5.Create();
byte[] md5Bytes = System.Text.Encoding.Default.GetBytes(value);
byte[] cryString = md5.ComputeHash(md5Bytes);
string md5Str = string.Empty;
for (int i = 0; i < cryString.Length; i++) {
md5Str += cryString[i].ToString("X");
}
return md5Str;
}
Call it with:
GetMD5(stringToConvert);