What I'm trying to do is process a downloaded image at a pixel level, in a vb6 console application, these are the steps currently
I download a png file from a website, this comes down as a 32 bit image, the image is grey scale, when the image is first
downloaded I need to convert it to a 24 bit Bitmap in order to be able to process it, if I open the image in Paint and save
it as a 24 bit bitmap I'm able to process the image without issue, when saving in Paint I dod get a message when saving that
says "Any Transparency will be lost if you save this picture.", Once saved, I then do a binary read on the file then extract the
BITMAPFILEHEADER
BITMAPINFOHEADER
BMPData
I can then loop through the BMPData and extract and process the needed value's. If I try to load the image without first saving
it to a 24 bit bitmap in paint, I receive a an out of memory error in VB (Run-Time Error 7) and the BMPInfoHeader.biSizeImage is
very big compaired to the size it is after with has been saved via paint.
I have tried to convert with ImageMagick-6.9.0-Q16 ad and GraphicsMagick-1.3.21-Q8 but these just seem to mess up the converted file.
Anyone got any idea's ?
Regards
Potman100
I have tried everything I know to get ImageMagick to create a 24-bit colour BMP from a greyscale 32-bit PNG file, including:
+matte to remove the alpha channel
-type truecolor to force a colour output
BMP4:output.bmp to force a BMP4
BMP3:output.bmp to force a BMP3
BMP2:output.bmp to force a BMP2
-colors 16|256 to force number of colours
-depth 8|24 to set the bit depth
passing through PPM format and back to get rid of PNG settings
writing via NetPBM's ppm2bmp/ppmtobmp
and all to no avail. I cannot get ImageMagick to create a 24-bit color BMP from a grey 32-bit PNG.
Update
I have worked out a horrible hack that will make you a 24-bit BMP, as follows. In a greyscale image, the Red=Green=Blue intensity for every pixel. If we decide to multiply every Blue pixel by 0.99, then, with just 1% error, the blue pixel will be less than the red and green, so ImageMagick will simply be unable to use a grayscale image to save the BMP in... like this:
convert input.png +matte -channel B -evaluate multiply 0.99 image.bmp
I don't know if that will work for you, if so, good, stop here. If not, continue...
The only thing I can suggest is that you use PPM format which is even easier than BMP for you to read from
convert yourImage.png output.ppm
That will get you a P6 type PPM image, as described here. If you want the even easier ASCII P3 version, use this command
convert yourImage.png -compress none output.ppm
and your image will look like this:
P3
384 128
255
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 255
Wow, thanks for all the time and effort you have taken to try and get a working solution for me,
I have tried the above and probably the one that will work is
convert yourImage.png -compress none output.ppm
I would have to do some further testing to check the values, but I think it would work.
The need to be able to create a 24bit bitmap is part of a project I am doing, the site I get
the png image from generates this image to offer a £ price for an item that I may sell to
them, so the png will contain a £ Value (i.e £ 1.56), I wanted to be able to work out the
value of the offer in a dos program I have written, so it can be called from php.
I did find a way to do it, I use xmlhttp to download the image via an adodb stream and found
that I could save the image in a number of different formats other than a bmp, so I looked
at the format of different image formats and found that the tif file format look promising
and found some code online that used Microsoft Image Acquistion Library to manipulate the tif
file.
By loading the tif file in to a WIA.ImageFile and applying a WIA.Vector to it, you can then
itterate the pixels in the image and get the raw color values, by adding the values together
by column and taking into account that you have column's that are all white, these just being
a blank column I was able to generate a unique value per character.
Thanks for all your help.
Potman100
Related
I have a vcf file in which ID is represented like this:
GTEX-1117F_GTEX-1117F
I want to edit the ID to look like:
only
GTEX-1117F
and exclude other part of the ID for all GTEX values.
Id GTEX-1117F_GTEX-1117F GTEX-111CU_GTEX-111CU GTEX-111FC_GTEX-111FC GTEX-111VG_GTEX-111VG GTEX-111YS_GTEX-111YS GTEX-1122O_GTEX-1122O GTEX-1128S_GTEX-1128S GTEX-113IC_GTEX-113IC
I tried to use this command:
sed -e '$s/\[[[:digit:]]\+\]//g; s/_GTEX[[:digit:]]\+//g' genotype_chr22_filtered_dosage2.txt > genotype_chr22_filtered_dosage3.txt
This is how my file looks like:
Id GTEX-1117F_GTEX-1117F GTEX-111CU_GTEX-111CU GTEX-111FC_GTEX-111FC GTEX-111VG_GTEX-111VG GTEX-111YS_GTEX-111YS GTEX-1122O_GTEX-1122O GTEX-
1128S_GTEX-1128S GTEX-113IC_GTEX-113IC GTEX-113JC_GTEX-113JC GTEX-117XS_GTEX-117XS GTEX-117YW_GTEX-117YW GTEX-117YX_GTEX-117YX GTEX-1192W_GTEX-11
92W GTEX-1192X_GTEX-1192X GTEX-11DXW_GTEX-11DXW GTEX-11DXX_GTEX-11DXX GTEX-11DXY_GTEX-11DXY GTEX-11DXZ_GTEX-11DXZ GTEX-11DYG_GTEX-11DYG GTEX-11
DZ1_GTEX-11DZ1 GTEX-11EI6_GTEX-11EI6 GTEX-11EM3_GTEX-11EM3 GTEX-11EMC_GTEX-11EMC GTEX-11EQ8_GTEX-11EQ8 GTEX-11EQ9_GTEX-11EQ9
GTEX-ZVZP_GTEX-ZVZP GTEX-ZVZQ_GTEX-ZVZQ GTEX-ZWKS_GTEX-ZWKS GTEX-ZXES_GTEX-ZXES GTEX-ZXG5_GTEX-ZXG5 GTEX-ZY6K_GTEX-ZY6K GTEX-ZYFC_GTEX-ZYFC GTEX-ZYFD_GTEX-ZYFD GTEX-ZYFG_GTEX-ZYFG GTEX-ZYT6_GTEX-ZYT6 GTEX-ZYVF_GTEX-ZYVF GTEX-ZYW4_GTEX-ZYW4 GTEX-ZYWO_GTEX-ZYWO
GTEX-ZYY3_GTEX-ZYY3 GTEX-ZZ64_GTEX-ZZ64 GTEX-ZZPT_GTEX-ZZPT GTEX-ZZPU_GTEX-ZZPU
22_20000146_G_A_b37 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0
This is part of my data
845000 0 0 0 0 0 0
845001 0 0 0 8 0 0
845002 1 0 0 0 0 0
845003 0 0 0 0 0 0
845004 0 0 7 0 0 0
845005 0 0 0 0 0 0
845006 0 0 0 0 0 0
845007 0 2 0 0 0 0
845008 0 0 0 0 0 11
845009 0 0 0 0 0 0
845010 0 0 7 0 0 0
845011 0 0 0 0 9 0
845012 0 0 0 0 0 0
845013 0 0 0 0 0 0
845014 0 0 0 8 0 0
845015 0 0 0 8 0 0
This is my picture
So I want to have all dots in black colour, but I can't and I don't know why. I write "pointcolor "black" and it doesn't work.
I use this in gnuplot
plot for [col=2:7] 'polaczone.txt' using 1:col with points pointtype 7 pointcolor "black" pointsize 2
I also have a question is it possible to delete showing "0" dots? I don't want them.
Use "linecolor" instead of "pointcolor".
How is this so complex or am I missing something. I simply want to get the integer value of a substring within an existing string and place it in a variable - same as php strpos.
The closest I have found is:
echo $haystack | awk '{print index($0,"<tagtosearch>")}';
Tried
myvar=$($haystack | awk '{print index($0,"<tagtosearch>")}');
but says command not found
The application is to automate include custom bash scripts on a given linux box, but not overwrite existing. Therefore I decided to insert custom start and end tags to denote the custom section. So I simply expected to get the start and end positions, delete this part of the file and pull in a latest version.
So this is as far as I have that function:
function install-env(){
mkdir -p /etc/datadimension/tmp;
cd /etc/datadimension/tmp;
cp /etc/bash.bashrc tempbash.bashrc;
newbash=$(cat tempbash.bashrc);
echo "$newbash" > newbash.bashrc;
insertstart=$(echo "$newbash" | awk '{print index($0,"<starttag>")}');
echo $insertstart;
}
Output
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Do I need to apt install something that can handle strings - this seems really basic requirement that is lacking.
With $($haystack), you invoke the value assigned to $haystack as a command.
To pipe $haystack value to awk try this:
myvar=$(echo "$haystack" | awk '{print index($0,"<tagtosearch>")}');
Or as #anubhava suggests in comment, using a here string:
myvar=$(awk '{print index($0, "<tagtosearch>")}' <<< "$haystack")
Could not find anything acceptable aside from php handling
If you want to print position of a search word in file then you can use this awk command:
awk -v kw='<starttag>' 'p=index($0, kw){p+=b; exit} {b+=length($0)+1} END{print p}' file
We keep adding each line's length into a running variable b until index retuns greater than 0. At that time we add current line's index into b and exit.
This one seems to work like php strpos() function
strpos() {
local string=$1
local findme=$2
awk -v x=$string -v y=$findme '{print index($0, y)}' <<< $string
}
Some tests
strpos azerty a
1
strpos azerty t
5
strpos azerty er
3
strpos azerty/uio /
7
strpos "azerty uio" " "
7
EDIT:
If you want exactly the php behaviour you can do:
php_strpos() {
local haystack="$1"
local needle="$2"
local offset=${3:-0}
php -r "echo strpos('$haystack', '$needle', $offset);"
}
I am trying to transform 8 columns of dummy variables into one column of a 8 level rank.
I am trying to do so with this formular:
=IF(OR(A1="1");"1";IF(OR(B1="1");"2";IF(OR(C1="1");"3";IF(OR(D1="1");"4";IF(OR(E1="1");"5";IF(OR(F1="1");"6";IF(OR(G1="1");"7";IF(OR(H1="1");"8";""))))))))
Here is a view on the table col. 1 to 8 is the data and col.9 is what I would like my command to return:
1 0 0 0 0 0 0 0 1
1 0 0 0 0 0 0 0 1
0 0 0 0 1 0 0 0 5
0 1 0 0 0 0 0 0 2
1 0 0 0 0 0 0 0 1
0 0 0 0 0 0 1 0 7
1 0 0 0 0 0 0 0 1
1 0 0 0 0 0 0 0 1
1 0 0 0 0 0 0 0 1
0 0 0 0 1 0 0 0 5
I have used these other stackoverflow questions as inspriration for the structure.
But it does not work, I don't get an error message, but I also don't get the right output.
Anyone who can see where the problem arises? - Would be much appreciated :)
Best wishes,
Mathilde
Use the MATCH() Function:
=MATCH(1,A1:H1,0)
It appears you use ; instead of , for the delimiter. If so use this.
=MATCH(1;A1:H1;0)
I am using GORM. My data field is varbinary(2000) but why is the data being inserted (new) being truncated to around 20 bytes?
var charData CharacterData
db.Where(CharacterData{CharId: charId, Segment: 0}).FirstOrInit(&charData)
charData.Data = data[16:]
db.Save(&charData)
len(data[16:]) is 2000.
type CharacterData struct {
CharId uint32
Data []byte
Segment uint8
SegmentId uint16
CreatedAt time.Time
UpdatedAt time.Time
DeletedAt time.Time
}
I also tried:
fmt.Println(string(charData0))
fmt.Println(len(charData0))
fmt.Println(len(string(charData0)))
0uyuyuyyuyuyuyy���������������������������� // only this gets inserted to the db which is short of the actual 1587
1587
1587
And it is also truncating the data. So I guess the problem lies with the to string casting.
The insert log shows:
INSERT INTO character_data (char_id, data, segment) VALUES ('2', '0tututututt����������������������������', '0')
data[16:] contains:
[0 0 48 6 1 0 1 0 104 101 104 101 104 101 104 101 104 101 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
update
Actual data inserted to database:
0tututututtÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
Instead of what normally should be something similar to:
pdukelÐÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ2ÿÿÿÿÿÿÿÿÿÿ1ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿHHllbÍfÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿúúúúúúúÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿþ ú
úúþ
þþþþÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ'ÿÿ
using raw queries
var charData0 []byte = data[16:]
hexData := fmt.Sprintf("%x", charData0)
fmt.Println(hexData)
db.Exec("INSERT INTO character_data (char_id, data, segment) VALUES (?, UNHEX(?), ?)", charId, hexData, 0)
=> 000030060100010079697969796979697979697900000000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
But the database only got:
0yiyiyiyiyyiyÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
update
Actually tried running the query on sqlpro:
INSERT INTO character_data (char_id, data, segment) VALUES ('4', UNHEX('000030060100010074757475747574757475740000000000050000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000'), '0')
And I still got truncated data. So does the problem lie with MYSQL?
EDIT:
You mentioned in the comment that you use MariaDB (not MySQL).
See MariaDB Tutorial: DATA TYPES:
VARBINARY(size) - Maximum size of 255 characters. - Where size is the number of characters to store. Variable-length string.
varbinary length is limited to 255 characters... Also it is in section STRING DATATYPES. This means MariaDB handles varbinary as a string...
This may very well explain the value you see: after about 255 bytes it contains just 0 values and nothing else, most likely because only 255 bytes come from the database and the rest is just filled with the zero-value of byte (which is 0).
A string in Go is stored as a byte array and interpreted using UTF-8 encoding.
A string may contain any byte sequence, but not all byte sequences represent valid UTF-8 encoded text.
So if you use Data to store arbitrary seuqence of bytes, you can't always display or treat it as a text. Just because printing the result of string(Data) conversion displays only 16 characters it doesn't mean Data contains only 16 bytes (or the bytes of 16 characters).
You can check it by printing its length:
fmt.Println(len(charData.Data))
Read this blog post to learn more about the topic:
Strings, bytes, runes and characters in Go
Edit:
You mentioned you use MySQL. What version?
MySQL version 5.0.2 and below only allows 255 bytes in varbinary, version 5.0.3 and above allows 64 KB. If the data inserted does not fit into the max (or defined) length, it is truncated.
Also please see MySQL 5.6 Reference Manual :: Section 11 Data Types :: 11.7 Data Type Storage Requirements:
The internal representation of a table has a maximum row size of 65,535 bytes, even if the storage engine is capable of supporting larger rows.
So if you (obviously) have other values in the row, they also have to fit into the row limit (which is 64 KB). Note that this is MySQL 5.6.