I use this Python code to upload to PageBlob by manually chunking the file. It works. I translated that logic to C#. I am wondering is there a simpler solution built into Azure SKD? less hacky solution.
byte[] bytes = (encoding ?? Encoding.UTF8).GetBytes(content);
int fileSize = bytes.Length;
int blockSize = fileSize;
int boundary = blockSize % 512;
if (boundary != 0)
{
byte[] padding = Enumerable.Repeat((byte)0x0, 512 - boundary).ToArray();
bytes = bytes.Concat(padding).ToArray();
blockSize = blockSize + 512 - boundary;
}
// Allocate pages
cloudPageBlob.Create(
blockSize,
accessCondition,
options,
operationContext);
var CHUNK_MAX_SIZE = 4 * 1024 * 1024;
var count = (int)Math.Ceiling(1.0 * bytes.Length / CHUNK_MAX_SIZE);
int remaining = bytes.Length;
for (var i = 0; i < count; i++)
{
int chunkSize = Math.Min(CHUNK_MAX_SIZE, remaining);
cloudPageBlob.UploadFromByteArray(
bytes,
i,
chunkSize,
accessCondition,
options,
operationContext);
remaining -= chunkSize;
}
I tried in my system able to upload chunks and slightly modify your code , taken dummy bytes to testing purpose
using System;
using System.Linq;
using System.Text;
using Microsoft.Azure.Storage;
using Microsoft.Azure.Storage.Auth;
using Microsoft.Azure.Storage.Blob;
namespace UploadPageChunck
{
class Program
{
static void Main(string[] args)
{
Uri uri = new Uri("blob uri");
Console.WriteLine("Hello World!");
AccessCondition data =null;
byte[] bytes= { 0x32, 0x00, 0x1E, 0x00 , 0x00 , 0x00 , 0x00 , 0x00 };
//byte[] bytes= { 0x32, 0x00, 0x1E, 0x00 , 0x00 , 0x00 , 0x00 , 0x00 , 0x00, 0x00, 0x00, };
var storage = new StorageCredentials("testsnapshotex", "Storage key ”)
int fileSize = bytes.Length;
int blockSize = fileSize;
int boundary = blockSize % 512;
if (boundary != 0)
{
byte[] padding = Enumerable.Repeat((byte)0x0, 512 - boundary).ToArray();
bytes = bytes.Concat(padding).ToArray();
blockSize = blockSize + 512 - boundary;
}
// Allocate pages
CloudPageBlob cloudPageBlob = new CloudPageBlob(uri,storage);
//cloudPageBlob.Create(
// blockSize,
// null,
// null,
// null);
var CHUNK_MAX_SIZE = 4 * 1024 * 1024;
var count = (int)Math.Ceiling(1.0 * bytes.Length / CHUNK_MAX_SIZE);
int remaining = bytes.Length;
for (var i = 0; i < count; i++)
{
int chunkSize = Math.Min(CHUNK_MAX_SIZE, remaining);
cloudPageBlob.UploadFromByteArray(
bytes,
i,
chunkSize,
null,
data,
null);
remaining -= chunkSize;
}
}
}
}
OUTPUT
Related
I'm using an Arduino Uno to build a smoke detection system, which works. Since I have to do an IoT project, I have to establish a secure connection with a (for example) Web server, and I'm using a Bluetooth module HC-05 to do it. Now I want to encrypt and decrypt messages (which contain the values of the gas sensor) using an AES library.
That's my code on Arduino IDE:
#include <AES.h>
#include <AESLib.h>
#include <AES_config.h>
#include <xbase64.h>
#define VCC2 5
int smokeA0 = A0;
int buzzer = 11;
float sensorValue;
void setup() {
pinMode(buzzer, OUTPUT);
pinMode(smokeA0, INPUT);
pinMode(VCC2, OUTPUT);
digitalWrite(VCC2, HIGH);
Serial.begin(9600);
Serial.println("Gas sensor warning up!");
delay(2000); //allow the sensor to warm up
noTone(buzzer);
}
void loop() {
sensorValue = analogRead(smokeA0);
Serial.print("Sensor value: ");
Serial.print(sensorValue);
if(sensorValue > 300){
Serial.print(" | Smoke detected!");
tone(buzzer,1000,2000);
}
else {
noTone(buzzer);
}
Serial.println("");
delay(200); //wait 2s for next reading
}
How is the code through which I can encrypt the value sensor? And have I include it to the code above?
In the site file zip containing the informations of the project there's also this file called "Receiver", on Arduino IDE, too:
#include <SoftwareSerial.h>
#include <LiquidCrystal.h>
#include <AES.h>
AES aes ;
byte key [N_BLOCK] ;
byte cipher [N_BLOCK] ;
byte check [N_BLOCK] ;
#define PIN_EN 6
#define BUTTON 13
SoftwareSerial btSerial(4, 5);
LiquidCrystal lcd(7, 8, 9, 10, 11, 12);
String msg = "";
bool isConnected = false;
String readString = "";
byte msg_received[16];
char final_str[16];
void setup( )
{
Serial.begin(9600);
Serial.print("Receiving Application\n\n");
lcd.begin(16, 2);
lcd.setCursor(0,0);
lcd.print("Receiving App");
int bits = 128;
set_bits (bits, key, 0) ; // all zero key
byte succ;
succ = aes.set_key (key, bits) ;
if (succ != SUCCESS) Serial.println ("Failure set_key") ;
checkConnection();
if(!isConnected) {
pinMode(PIN_EN, OUTPUT); // this pin will pull the HC-010 EN HIGH to switch module to AT mode
digitalWrite(PIN_EN, HIGH);
Serial.println("Going in AT mode for BLE HC-10");
btSerial.begin(38400);
delay(1000);
pinMode(A0, INPUT);
while(!isConnected) {
if (btSerial.available()) {
char c = btSerial.read();
Serial.write(c);
}
if (Serial.available()) {
char c = Serial.read();
btSerial.write(c);
}
checkConnection();
}
btSerial.flush();
btSerial.end();
btSerial.begin(9600);
digitalWrite(PIN_EN, LOW);
Serial.println("Exiting from AT mode for BLE HC-10");
}
}
void checkConnection() {
int x = analogRead(A0);
if(x > 700) {
Serial.println("HC-10 is connected");
isConnected = true;
}
}
void loop( )
{
if(isConnected) {
getBTReply();
if(msg.length() > 0) {
Serial.println("Received msg: " + msg);
lcd.clear();
lcd.begin(16, 2);
lcd.setCursor(0,0);
lcd.print("Receiving App");
lcd.setCursor(0, 1);
lcd.print("Temp. > ");
lcd.print(msg);
lcd.setCursor(13, 1);
lcd.print(" C");
} else {
Serial.println("Received no msg");
lcd.clear();
lcd.begin(16, 2);
lcd.setCursor(0,0);
lcd.print("Receiving App");
lcd.setCursor(0, 1);
lcd.print("no message received");
}
}
delay(4000);
}
void getBTReply() {
msg = "";
int i = 0;
while (btSerial.available()) {
msg_received[i] = btSerial.read();
i++;
}
if (i==16) {
byte succ;
succ = aes.decrypt (msg_received, check) ;
if (succ != SUCCESS) Serial.println ("Failure decrypt") ;
String x = prs_byte_hex(check, 128);
textFromHexString(x.c_str(), final_str);
msg = String(final_str);
memset(final_str,0,strlen(final_str));
}
}
void set_bits (int bits, byte * a, int count)
{
bits >>= 3 ;
byte bcount = count >> 3 ;
for (byte i = 0 ; i < bcount ; i++)
a [i] = 0xFF ;
if ((count & 7) != 0)
a [bcount++] = 0xFF & (0xFF00 >> (count & 7)) ;
for (byte i = bcount ; i < bits ; i++)
a [i] = 0x00 ;
}
char * hex = "0123456789abcdef" ;
void print_value (byte * a, int bits)
{
bits >>= 3 ;
for (int i = 0 ; i < bits ; i++)
{
byte b = a[i] ;
// test purpose only
Serial.print (hex [b >> 4]) ;
Serial.print (hex [b & 15]) ;
}
Serial.println () ;
}
String prs_byte_hex (byte * a, int bits)
{
bits >>= 3 ;
String str_toparse;
for (int i = 0 ; i < bits ; i++)
{
byte b = a[i] ;
str_toparse += hex [b >> 4];
str_toparse += hex [b & 15];
}
return str_toparse;
}
void textFromHexString(char *hex, char *result)
{
char temp[3];
int index = 0;
temp[2] = '\0';
while (hex[index])
{
strncpy(temp, &hex[index], 2);
*result = (char)strtol(temp, NULL, 16);
result++;
index += 2;
}
*result = '\0';
}
So I don't need to build a Web Server in VSC to establish a connection through two devices?
If yes, can you please share me the code?
Sorry, I'm not very practical in IoT and I'm completely alone to do the project.
If you're sending data using the HC-05 to another arduino, then here is how to encrypt the data:
uint8_t key[] = {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15};
char data[] = "0123456789012345"; //16 chars == 16 bytes - Your data you want to send
aes128_enc_single(key, data);
Then on the receiving arduino
aes128_dec_single(key, data);
You send that data using serial. The receiving Arduino would also need an HC-05 module, or if you're sending to your computer, you can use a USB bluetooth adapter. You don't need a webserver to establish a connection between 2 arduinos, no. If you're sending data online, then you'll need to have a way to connect to the internet from one of your devices.
I have video file (mp4). I want to detach audio stream (AAC format) from that file and save in PC.
With below code, Generated aac file canplay now on KM player, but can not play on VLC player. Information of duration displays on player is wrong.
Please help me with this problem.
err = avformat_open_input(input_format_context, filename, NULL, NULL);
if (err < 0) {
return err;
}
/* If not enough info to get the stream parameters, we decode the
first frames to get it. (used in mpeg case for example) */
ret = avformat_find_stream_info(*input_format_context, 0);
if (ret < 0) {
av_log(NULL, AV_LOG_FATAL, "%s: could not find codec parameters\n", filename);
return ret;
}
/* dump the file content */
av_dump_format(*input_format_context, 0, filename, 0);
for (size_t i = 0; i < (*input_format_context)->nb_streams; i++) {
AVStream *st = (*input_format_context)->streams[i];
if (st->codec->codec_type == AVMEDIA_TYPE_AUDIO) {
*input_codec_context = st->codec;
*input_audio_stream = st;
FILE *file = NULL;
file = fopen("C:\\Users\\MyPC\\Downloads\\Test.aac", "wb");
AVPacket reading_packet;
av_init_packet(&reading_packet);
while (av_read_frame(*input_format_context, &reading_packet) == 0) {
if (reading_packet.stream_index == (int) i) {
uint8_t adts_header[7];
unsigned int obj_type = 0;
unsigned int num_data_block = (reading_packet.size)/1024;
int rate_idx = st->codec->sample_rate, channels = st->codec->channels;
uint16_t frame_length;
// include the header length also
frame_length = reading_packet.size + 7;
/* We want the same metadata */
/* Generate ADTS header */
if(adts_header == NULL) return -1;
/* Sync point over a full byte */
adts_header[0] = 0xFF;
/* Sync point continued over first 4 bits + static 4 bits
* (ID, layer, protection)*/
adts_header[1] = 0xF1;
/* Object type over first 2 bits */
adts_header[2] = obj_type << 6;
/* rate index over next 4 bits */
adts_header[2] |= (rate_idx << 2);
/* channels over last 2 bits */
adts_header[2] |= (channels & 0x4) >> 2;
/* channels continued over next 2 bits + 4 bits at zero */
adts_header[3] = (channels & 0x3) << 6;
/* frame size over last 2 bits */
adts_header[3] |= (frame_length & 0x1800) >> 11;
/* frame size continued over full byte */
adts_header[4] = (frame_length & 0x1FF8) >> 3;
/* frame size continued first 3 bits */
adts_header[5] = (frame_length & 0x7) << 5;
/* buffer fullness (0x7FF for VBR) over 5 last bits*/
adts_header[5] |= 0x1F;
/* buffer fullness (0x7FF for VBR) continued over 6 first bits + 2 zeros
* number of raw data blocks */
adts_header[6] = 0xFA;
adts_header[6] |= num_data_block & 0x03; // Set raw Data blocks.
fwrite(adts_header, 1, 7, file);
fwrite(reading_packet.data, 1, reading_packet.size, file);
}
av_free_packet(&reading_packet);
}
fclose(file);
return 0;
}
}
Object type and sample rate index must be set to the real, correct values. Both values can be parsed out of the audio specific config in the extradata field in the codec context. All the information you need is here: http://wiki.multimedia.cx/index.php?title=MPEG-4_Audio
I am trying to read and write to the AT24MAC402 EEPROM over i2c on the Cubieboard 2 using Arch Linux. I am using the i2c-dev library and i2c-tools.
Datasheet:
http://www.atmel.com/images/atmel-8807-seeprom-at24mac402-602-datasheet.pdf
I can successfully write (kind of...) to a chosen address and sequentially write many bites starting at that address. The issues are:
Cannot re-select another address to write once the first address has been selected.
Cannot point the the EEPROM to the location I wish to read from (by dummy-writing), and therefore have almost no real control over the EEPROM.
Upon looking at the datasheet (for hours on end), it looks as if I don't have as much control over the I2C communications as I may need using the i2c-dev library.. It would be great if I could just write X bits or X bytes directly to the EEPROM.
In short, I would like advice on how I can read and write properly to this EEPROM.
char buf[10];
int com_serial;
int failcount;
int i2c_init(char filename[40], int addr)
{
int file;
if ((file = open(filename,O_RDWR)) < 0)
{
printf("Failed to open the bus.");
/* ERROR HANDLING; you can check errno to see what went wrong */
com_serial=0;
exit(1);
}
if (ioctl(file,I2C_SLAVE,addr) < 0)
{
printf("Failed to acquire bus access and/or talk to slave.\n");
/* ERROR HANDLING; you can check errno to see what went wrong */
com_serial=0;
exit(1);
}
return file;
}
int main (int argc, char *argv[]) {
char read_buf[16];
char write_buf[17];
int i;
int file;
file=i2c_init("/dev/i2c-1",0x50); //dev,slavei2caddr
write_buf[0] = 0x00;
write_buf[1] = 'H';
write_buf[2] = 'i';
write_buf[3] = '!';
write(file, write_buf, 4);
//Successfully prints "Hi!" to bytes 0x00 -> 0x02
//Setting EEPROM to point to address 0xA0 to start reading (arbitrary address with known values: all 0xFF)
write_buf[0] = 0xA0;
write(file, write_buf, 1);
//Reading 1 byte from EEPROM, even though there is a '2'; 2 bytes would be '3'
read(file, read_buf, 2);
for (i=1; i<3; i++){
printf("%X", read_buf[i]);
}
//Prints out from address 0x04 to 0x05 instead of 0xA0 to 0xA1
printf("\n");
}
I did work properly using the functions from the linux/i2c-dev.h.
To test the code I get the output generated by i2cdump and put as input to i2c-stub-from-dump tool, it lets you setup one or more fake I2C chips on the i2c-stub bus based on dumps of the chips you want to emulate.
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <linux/i2c-dev.h>
int i2c_init(const char * i2c_device, const int chip_address)
{
int file;
if ((file = open(i2c_device, O_RDWR)) < 0) {
return -1;
}
if (ioctl(file, I2C_SLAVE, chip_address) < 0) {
close(file);
return -1;
}
return file;
}
int i2c_write(int file, const int data_address, const unsigned char * data, size_t size)
{
return i2c_smbus_write_i2c_block_data(file, data_address, size, data);
}
void i2c_read(int file, const int data_address, unsigned char * data_vector, size_t size)
{
unsigned char reg = data_address;
unsigned int i;
for(i = 0; i < size; ++i, ++reg) {
data_vector[i] = i2c_smbus_read_byte_data(file, reg);
}
}
int main(void) {
char device[] = "/dev/i2c-6";
int address = 0x50;
unsigned char buffer_before[30] = {0};
unsigned char buffer_after[30] = {0};
unsigned char data[] = "Hello World!";
int file;
file = i2c_init(device, address);
if (file > 0) {
i2c_read(file, 0x00, buffer_before, sizeof(data));
i2c_write(file, 0x00, data, sizeof(data));
i2c_read(file, 0x00, buffer_after, sizeof(data));
close (file);
}
printf("data read before write: %s\n", buffer_before);
printf("data read after write: %s\n", buffer_after);
return 0;
}
I am new to WASAPI, following the msdn reference code, here,
http://msdn.microsoft.com/en-us/library/windows/desktop/dd370800(v=vs.85).aspx, to capture audio using WASAPI apis.
Modified msdn reference code slightly for my purpose. I am using a microphone to record my voice, and play it back, it works fine when using SoundRecorder and other Windows in-built apps, but using my test application, not getting any valid sound, just getting some noise.
Here is my code, please let me know, where could I be going wrong:
// REFERENCE_TIME time units per second and per millisecond
#define REFTIMES_PER_SEC 10000000
#define REFTIMES_PER_MILLISEC 10000
#define TIME_COUNTER_LIMIT 20
WAVEFORMATEX sinWaveFormat;
CWaveFile sinwave;
HRESULT RecordAudioStream()
{
HRESULT hr;
REFERENCE_TIME hnsRequestedDuration = REFTIMES_PER_SEC;
REFERENCE_TIME hnsActualDuration;
UINT32 bufferFrameCount;
UINT32 numFramesAvailable;
IMMDeviceEnumerator *pEnumerator = NULL;
IMMDevice *pDevice = NULL;
IAudioClient *pAudioClient = NULL;
IAudioCaptureClient *pCaptureClient = NULL;
WAVEFORMATEX *pwfx = NULL;
UINT32 packetLength = 0;
UINT32 time_counter = 0;
BYTE *pData;
DWORD flags;
UINT32 bytesToCapture = 0;
UINT64 u64DevicePosition = 0;
UINT64 u64QPCPosition = 0;
BYTE temp_buffer[10000];
CoInitializeEx(NULL, COINIT_MULTITHREADED);
hr = CoCreateInstance(
__uuidof(MMDeviceEnumerator), NULL,
CLSCTX_ALL, __uuidof(IMMDeviceEnumerator),
(void**)&pEnumerator);
EXIT_ON_ERROR(hr)
hr = pEnumerator->GetDefaultAudioEndpoint(
eCapture, eConsole, &pDevice);
EXIT_ON_ERROR(hr)
hr = pDevice->Activate(
__uuidof(IAudioClient), CLSCTX_ALL,
NULL, (void**)&pAudioClient);
EXIT_ON_ERROR(hr)
hr = pAudioClient->GetMixFormat(&pwfx);
EXIT_ON_ERROR(hr)
// convert from Float to PCM and from WAVEFORMATEXTENSIBLE to WAVEFORMATEX
if ((pwfx->wFormatTag == WAVE_FORMAT_IEEE_FLOAT) ||
((pwfx->wFormatTag == WAVE_FORMAT_EXTENSIBLE) &&
(reinterpret_cast<WAVEFORMATEXTENSIBLE *>(pwfx)->SubFormat == KSDATAFORMAT_SUBTYPE_IEEE_FLOAT)))
{
pwfx->wFormatTag = WAVE_FORMAT_PCM;
pwfx->wBitsPerSample = 16;
pwfx->nBlockAlign = pwfx->nChannels * 2; // (nChannels * wBitsPerSample) / 8
pwfx->nAvgBytesPerSec = pwfx->nSamplesPerSec * pwfx->nBlockAlign;
pwfx->cbSize = 0;
}
hr = open_capture_file(pwfx);
EXIT_ON_ERROR(hr)
hr = pAudioClient->Initialize(
AUDCLNT_SHAREMODE_SHARED,
0,
hnsRequestedDuration,
0,
pwfx,
NULL);
EXIT_ON_ERROR(hr)
// Get the size of the allocated buffer.
hr = pAudioClient->GetBufferSize(&bufferFrameCount);
EXIT_ON_ERROR(hr)
hr = pAudioClient->GetService(
__uuidof(IAudioCaptureClient),
(void**)&pCaptureClient);
EXIT_ON_ERROR(hr)
/*
// Notify the audio sink which format to use.
hr = pMySink->SetFormat(pwfx);
EXIT_ON_ERROR(hr)
*/
// Calculate the actual duration of the allocated buffer.
hnsActualDuration = (double)REFTIMES_PER_SEC *
bufferFrameCount / pwfx->nSamplesPerSec;
hr = pAudioClient->Start(); // Start recording.
EXIT_ON_ERROR(hr)
// Sleep for half the buffer duration.
Sleep(hnsActualDuration / REFTIMES_PER_MILLISEC / 2);
hr = pCaptureClient->GetNextPacketSize(&packetLength);
EXIT_ON_ERROR(hr)
bytesToCapture = packetLength * pwfx->nBlockAlign;
while (packetLength != 0 && time_counter <= TIME_COUNTER_LIMIT)
{
time_counter++;
// Get the available data in the shared buffer.
hr = pCaptureClient->GetBuffer(
&pData,
&numFramesAvailable,
&flags, &u64DevicePosition, &u64QPCPosition);
EXIT_ON_ERROR(hr)
if (packetLength != numFramesAvailable)
{
printf("packetlength = %d, numFramesAvailable = %d, does not match.\n", packetLength, numFramesAvailable);
bytesToCapture = numFramesAvailable * pwfx->nBlockAlign;
}
printf("packetlength = %d, numFramesAvailable = %d, bytesToCapture = %d.\n",
packetLength, numFramesAvailable, bytesToCapture);
if (flags & AUDCLNT_BUFFERFLAGS_SILENT)
{
memset(pData, 0, numFramesAvailable * pwfx->nBlockAlign);
}
if (bytesToCapture > sizeof(temp_buffer))
{
printf("bytesToCapture = %d, more than buffer size = %d\n.", bytesToCapture, sizeof(temp_buffer));
continue;
}
memcpy(temp_buffer, pData, bytesToCapture);
hr = pCaptureClient->ReleaseBuffer(numFramesAvailable);
EXIT_ON_ERROR(hr)
// Copy the available capture data to the audio sink.
hr = write_to_file(
temp_buffer, bytesToCapture);
EXIT_ON_ERROR(hr)
// Sleep for half the buffer duration.
//Sleep(hnsActualDuration / REFTIMES_PER_MILLISEC / 2);
hr = pCaptureClient->GetNextPacketSize(&packetLength);
EXIT_ON_ERROR(hr)
bytesToCapture = packetLength * pwfx->nBlockAlign;
}
hr = pAudioClient->Stop(); // Stop recording.
EXIT_ON_ERROR(hr)
Exit:
CoTaskMemFree(pwfx);
SAFE_RELEASE(pEnumerator);
SAFE_RELEASE(pDevice);
SAFE_RELEASE(pAudioClient);
SAFE_RELEASE(pCaptureClient);
return hr;
}
I have here the following data for which I have to find the sha1 digest using openssl.
data:
AwAIAOwIAAABABwAgAIAABYAAAAAAAAAAAAAAHQAAAAAAAAAAAAAAAgAAAAkAAAAQgAAAFQAAABsAAAAhgAAAJgAAACuAAAAwgAAAM4AAADsAAAAAgEAAAwBAAAoAQAARgEAAFgBAACwAQAAtAEAANABAADkAQAA+gEAAAIAaQBkAAAADABsAGEAeQBvAHUAdABfAHcAaQBkAHQAaAAAAA0AbABhAHkAbwB1AHQAXwBoAGUAaQBnAGgAdAAAAAcAZwByAGEAdgBpAHQAeQAAAAoAYgBhAGMAawBnAHIAbwB1AG4AZAAAAAsAbwByAGkAZQBuAHQAYQB0AGkAbwBuAAAABwBwAGEAZABkAGkAbgBnAAAACQB0AGUAeAB0AEMAbwBsAG8AcgAAAAgAdABlAHgAdABTAGkAegBlAAAABAB0AGUAeAB0AAAADQBwAGEAZABkAGkAbgBnAEIAbwB0AHQAbwBtAAAACQBzAGMAYQBsAGUAVAB5AHAAZQAAAAMAcwByAGMAAAAMAHAAYQBkAGQAaQBuAGcAUgBpAGcAaAB0AAAADQBsAGEAeQBvAHUAdABfAHcAZQBpAGcAaAB0AAAABwBhAG4AZAByAG8AaQBkAAAAKgBoAHQAdABwADoALwAvAHMAYwBoAGUAbQBhAHMALgBhAG4AZAByAG8AaQBkAC4AYwBvAG0ALwBhAHAAawAvAHIAZQBzAC8AYQBuAGQAcgBvAGkAZAAAAAAAAAAMAEwAaQBuAGUAYQByAEwAYQB5AG8AdQB0AAAACABUAGUAeAB0AFYAaQBlAHcAAAAJAEkAbQBhAGcAZQBWAGkAZQB3AAAABgBCAHUAdAB0AG8AbgAAAAAAgAEIAEQAAADQAAEB9AABAfUAAQGvAAEB1AABAcQAAQHVAAEBmAABAZUAAQFPAQEB2QABAR0BAQEZAQEB2AABAYEBAQEAARAAGAAAABEAAAD/////DwAAABAAAAACARAAsAAAABEAAAD//////////xIAAAAUABQABwAAAAAAAAAQAAAAAwAAAP////8IAAAREQAAABAAAAAFAAAA/////wgAABABAAAAEAAAAAAAAAD/////CAAAAR0AB38QAAAABAAAAP////8IAAABEQAGfxAAAAAGAAAA/////wgAAAUBEAAAEAAAAAEAAAD/////CAAAEP////8QAAAAAgAAAP////8IAAAQ/////wIBEACcAAAAGgAAAP//////////EwAAABQAFAAGAAAAAAAAABAAAAAIAAAA/////wgAAAUCEgAAEAAAAAcAAAD/////CAAAARAABn8QAAAACgAAAP////8IAAAFARgAABAAAAABAAAA/////wgAABD/////EAAAAAIAAAD/////CAAAEP7///8QAAAACQAAAP////8IAAABRwAIfwMBEAAYAAAAIAAAAP//////////EwAAAAIBEAB0AAAAIgAAAP//////////EgAAABQAFAAEAAAAAAAAABAAAAAFAAAA/////wgAABAAAAAAEAAAAAQAAAD/////CAAAAREABn8QAAAAAQAAAP////8IAAAQ/////xAAAAACAAAA/////wgAABD+////AgEQAIgAAAAoAAAA//////////8UAAAAFAAUAAUAAAAAAAAAEAAAAA0AAAD/////CAAABQEYAAAQAAAAAQAAAP////8IAAAQ/v///xAAAAACAAAA/////wgAABD+////EAAAAAwAAAD/////CAAAAQEAAn8QAAAACwAAAP////8IAAAQBQAAAAMBEAAYAAAALQAAAP//////////FAAAAAIBEAB0AAAALwAAAP//////////EgAAABQAFAAEAAAAAAAAABAAAAAFAAAA/////wgAABABAAAAEAAAAAEAAAD/////CAAABQEAAAAQAAAAAgAAAP////8IAAAQ/v///xAAAAAOAAAA/////wgAAAQAAIA/AgEQAHQAAAA1AAAA//////////8VAAAAFAAUAAQAAAAAAAAAEAAAAAAAAAD/////CAAAASgAB38QAAAAAQAAAP////8IAAAQ/////xAAAAACAAAA/////wgAABD+////EAAAAAkAAAD/////CAAAARUACH8DARAAGAAAADgAAAD//////////xUAAAACARAAdAAAADoAAAD//////////xUAAAAUABQABAAAAAAAAAAQAAAAAAAAAP////8IAAABKgAHfxAAAAABAAAA/////wgAABD/////EAAAAAIAAAD/////CAAAEP7///8QAAAACQAAAP////8IAAABGgAIfwMBEAAYAAAAPQAAAP//////////FQAAAAMBEAAYAAAAPwAAAP//////////EgAAAAIBEAB0AAAAQQAAAP//////////EgAAABQAFAAEAAAAAAAAABAAAAAFAAAA/////wgAABABAAAAEAAAAAEAAAD/////CAAABQEAAAAQAAAAAgAAAP////8IAAAQ/v///xAAAAAOAAAA/////wgAAAQAAIA/AgEQAHQAAABHAAAA//////////8VAAAAFAAUAAQAAAAAAAAAEAAAAAAAAAD/////CAAAASkAB38QAAAAAQAAAP////8IAAAQ/////xAAAAACAAAA/////wgAABD+////EAAAAAkAAAD/////CAAAARYACH8DARAAGAAAAEoAAAD//////////xUAAAACARAAdAAAAEwAAAD//////////xUAAAAUABQABAAAAAAAAAAQAAAAAAAAAP////8IAAABKwAHfxAAAAABAAAA/////wgAABD/////EAAAAAIAAAD/////CAAAEP7///8QAAAACQAAAP////8IAAABGQAIfwMBEAAYAAAATwAAAP//////////FQAAAAMBEAAYAAAAUQAAAP//////////EgAAAAMBEAAYAAAAUwAAAP//////////EgAAAAMBEAAYAAAAVQAAAP//////////EgAAAAEBEAAYAAAAVQAAAP////8PAAAAEAAAABgAAAA9AAAA//////////8fAAAAAgEQAGAAAAA/AAAA//////////8eAAAAFAAUAAMAAAAAAAAAGQAAAAUAAAD/////CAAAEAAAAAAZAAAAAAAAAP////8IAAAQ/v///xkAAAABAAAA/////wgAABD+////AgEQAMQAAABEAAAA//////////8gAAAAFAAUAAgAAAAAAAAAGQAAABIAAAD/////CAAABQIOAAAZAAAAEQAAAP////8IAAARAQAAABkAAAAQAAAA/////wgAAAEGAAZ/GQAAAAIAAAD/////CAAAARIAB38ZAAAAEwAAAP////8IAAAFAQQAABkAAAAAAAAA/////wgAABD+////GQAAAAEAAAD/////CAAAEP7///8ZAAAADwAAAP////8IAAABMwAIfwMBEAAYAAAASwAAAP//////////IAAAAAIBEACIAAAATQAAAP//////////IAAAABQAFAAFAAAAAAAAABkAAAASAAAA/////wgAAAUCDgAAGQAAABAAAAD/////CAAAAQYABn8ZAAAAAgAAAP////8IAAABEwAHfxkAAAAAAAAA/////wgAABD+////GQAAAAEAAAD/////CAAAEP7///8DARAAGAAAAFEAAAD//////////yAAAAADARAAGAAAAFMAAAD//////////x4AAAACARAAYAAAAFUAAAD//////////x4AAAAUABQAAwAAAAAAAAAZAAAABQAAAP////8IAAAQAAAAABkAAAAAAAAA/////wgAABD+////GQAAAAEAAAD/////CAAAEP7///8CARAAxAAAAFoAAAD//////////yAAAAAUABQACAAAAAAAAAAZAAAAEgAAAP////8IAAAFAg4AABkAAAARAAAA/////wgAABEBAAAAGQAAABAAAAD/////CAAAAQYABn8ZAAAAAgAAAP////8IAAABFAAHfxkAAAATAAAA/////wgAAA
The digest as given to me is:
Wk2pJnOErEHsElMw4TMX+rjHsQQ=
But when I use(f1= file where I copied the above data):
base64 -d f1.txt | openssl dgst -sha1 -binary | base64
I get a "base64: invalid input" error and the following digest which seems completely different :(
BaRlDid73RYBFMgqveC8G+gFBBU=
Can somebody confirm and explain if there is some mistake??
UPDATED:
Scenario: Client's binary file is base64 encoded and sent to server. Server decodes this and computes the sha1 digest. Since I have client's base64 encoded sha1 digest, the server also encodes the digest to base64. Now these two should match. And it doesn't! I receive all data. I have rechecked it. I shall present part of the code here:
//RCVBUFSIZE = 1024 (defined)
void HandleClient(int clntSocket)
{
char echoBuffer[RCVBUFSIZE] ; /* Buffer for echo string */
memset(echoBuffer, 0, RCVBUFSIZE);
char inBuffer; /* Buffer for first string */
char recv_data;
int recvMsgSize = 0; /* Size of received message */
char replyBuffer[32];
int bytes_received = 0;
int rv = 0;
int connected = clntSocket;
int len= 0;
int i = 0;
EVP_MD_CTX md_ctx;
const EVP_MD *md;
unsigned char md_value[EVP_MAX_MD_SIZE];
unsigned int md_len;
OpenSSL_add_all_digests();
md = EVP_get_digestbyname("sha1");
EVP_MD_CTX_init(&md_ctx);
EVP_DigestInit_ex(&md_ctx, md, NULL);
/* Receive message from client */
while (((bytes_received = recv(connected,&inBuffer,1,0)) > 0) && (inBuffer != '\n')){
/* Send received string and receive again until end of transmission */
if (bytes_received > 0) /* zero indicates end of transmission */
{
printf("Message received from Client is : %c\n", inBuffer);
char n = inBuffer;
int indicator = 0;
int current = 0;
unsigned long fileLen;
if(n =='6'){
if ((recvMsgSize = recv(connected, echoBuffer, RCVBUFSIZE, 0)) < 0)
DieWithError("recv() failed");
printf("no. of bytes got : %d\n", recvMsgSize);
if (recvMsgSize > 0)
echoBuffer[recvMsgSize] = '\0';
len= atoi(echoBuffer);
char *data =NULL;
printf("length of following message : %d\n", len);
if(len>0){
for( i = RCVBUFSIZE; i < (len+RCVBUFSIZE); i=i+RCVBUFSIZE){
if(i>len)
recvMsgSize = recv(connected, echoBuffer, (len - (i-RCVBUFSIZE)), 0);
else
recvMsgSize = recv(connected, echoBuffer, RCVBUFSIZE, 0);
echoBuffer[recvMsgSize] = '\0';
decode(echoBuffer, recvMsgSize, "file_out");
data = readFileBuffer("file_out");
EVP_DigestUpdate(&md_ctx, data, strlen(data));
}
}
len = 0;
memset(echoBuffer, 0, RCVBUFSIZE);
recvMsgSize = 0;
}
if (n =='5'){
printf("Update Digest Over- Calculate Final Dgst!!!!! \n");
n= 0;
EVP_DigestFinal_ex(&md_ctx, md_value, &md_len); //retrieve digest from ctx unto md_value and #bytes written is copied into md_len
EVP_MD_CTX_cleanup(&md_ctx);
FILE *f;
f = fopen("file_sha1", "w");
printf("\n");
printf("******************************************************\n ");
printf("Digest is: ");
for(i = 0; i < md_len; i++){
if ( f !=NULL){
fputc(md_value[i], f);
}
printf("%02x", md_value[i]);
}
printf("\n");
printf("******************************************************\n ");
fclose(f);
}
printf("socket closing\n");
close(connected); /* Close client socket */
}
}
char *readFileBuffer(char *name)
{
FILE *file;
char *buffer = NULL;
unsigned long fileLen;
//Open file
file = fopen(name, "rb");
if (!file)
{
fprintf(stderr, "Unable to open file %s", name);
return;
}
//Get file length
fseek(file, 0, SEEK_END);
fileLen=ftell(file);
printf("file length = %ld\n", fileLen);
fseek(file, 0, SEEK_SET);
//printf("Allocate memory\n");
buffer=(char *)malloc(fileLen+1);
printf("length of write buffer = %d\n", strlen(buffer));
if (!buffer)
{
fprintf(stderr, "Memory error!");
}
long int n = fread(buffer,1, fileLen,file);
buffer[n] = '\0';
printf("Read no. of bytes = %ld into buffer \n", n);
printf("len of buffer %d \n", strlen(buffer));
if (!buffer)
{
fprintf(stderr, "Memory error!");
fclose(file);
}
fclose(file);
//free(name);
return buffer;
}
// reads b64 encoded msg (ReadBuffer) and writes to WriiteFile.
void decode(char *ReadBuffer, int Length, char *WriteFile)
{
char *msg = (char *)malloc(Length);
memset(msg, 0x00, Length);
int readbytes = -1;
printf("buffer write file %s\n", WriteFile);
// the decode msg is written to this bio
BIO *fileWrBIO = BIO_new_file(WriteFile, "w");
BIO *b64 = BIO_new(BIO_f_base64());
BIO *bio = BIO_new_mem_buf(ReadBuffer, Length);
bio = BIO_push(b64, bio);
BIO_set_flags(bio,BIO_FLAGS_BASE64_NO_NL);
while ((readbytes = BIO_read(bio, msg, Length)) > 0)
{
printf("readbytes: %d\n", readbytes);
BIO_write(fileWrBIO, msg, readbytes);
BIO_flush(fileWrBIO);
memset(msg, 0x00, sizeof(msg));
}
free(msg);
BIO_free_all(bio);
BIO_free_all(fileWrBIO);
}
FWIW...
There are implementations where base64 cannot read its own output.
# base64 ssh_host_rsa_key | base64 -d
-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEA7qHASF1Jgbase64: invalid input
This is on a CentOS 5 machine.
Reason is that base64 produces output with line breaks, which are garbage chars for the decoder.
Solution is to either produce the base64 without linebreaks (-w 0) or make the decoder ignore garbage chars (-i).
Your data is invalid, probably partial. A valid base64 encoded string should have a length multiple of 4. So the different digest output is expected.
You can encrypt using this command
base64 -w 0 < id_rsa
Well, the data doesn't seem to be a valid base64 string. You might be missing some characters.
Just hit this, also on CentOS 5. Both -w 0 and -i were required, -i didn't work alone. e.g.:
tar -cf - /home/backup | gzip | base64 -w 0
base64 -d -i | gunzip | tar -xvf - -C /
worked fine to move a small home directory via copy&paste.