I'm trying to use openAI's CLIP tool, to do a simple thing - trying to guess how much a picture is close to a label.
When I tried to run the model on 1 dog image, I used the labels:
labels = ["a diagram", "a dog", "a cat"]
which proves to be really good (I hope) since I got 0.996 for the dog option.
But since I want to get a guess on a specific label, I tried the labels:
labels = ["dog", "other"]
which looks good - I got 0.9988 for the dog option, And when I tried to run it again on a bird image, I got 0.0199 for the dog label (and 0.98 for other).
So. Is there a better method for the "other" label?
(the code can be found here - https://github.com/openai/CLIP. I've used openAI first example, with file and label tweaks).
Related
Object subclass: #MultiData
instanceVariableNames: 'b'
classVariableNames: ''
package: 'CFR-Extensions'
initialize
b := RTGrapher new.
b add: (self makeD: #('hello' 1 2 1)).
b add: (self makeD: #('test' 1 2 11)).
b
makeD: first
| d |
d := RTVerticalMultipleData new.
d barShape color: Color blue.
points := OrderedCollection new.
points add: first.
d points: points.
d addMetric: #second.
d addMetric: #third.
d addMetric: #fourth.
"Rotated text"
d barChartWithBarTitle: #first rotation: -30.
^d
The above is essentially the Several metrics per data point example from the Roassal book factored into two methods. Rather than just visualizing a static dataset I've been looking into ways of trying to add data as the program runs. I want to visualize the trace of the parameters for a tabular RL agent.
What happens when I display the graph in the inspector is that only the latest element shows up as a chart. There is some overlaying in the labels though that should not be there.
Originally I wanted to do something like pass an OrderedCollection of points, but the way RTVerticalMultipleData compiles them into Trachel elements makes such a scheme invalid, so I've thought to batch the data instead before adding it as an element.
The fact that the above does not work strikes me as a bug. Apart from fixing this, I am wondering if there is a better way to visualize dynamic data?
I don't know roassal enough to answer to your problem, but for dynamic visualizations, Pharo also has the Telescope project. (https://github.com/TelescopeSt/Telescope)
Currently, Telescope only works with Seaside via web visualization (With the Cytoscape connector: https://github.com/TelescopeSt/TelescopeCytoscape). See a demo at: https://demos.ferlicot.fr/TelescopeDemo
I don't know if web visualizations are fine with you but I share just in case.
I am able to build the model using the built-in lee_background corpus. But when I try to compare using most_similar method, I get an error.
lee_train_file = '/opt/conda/lib/python3.6/site-packages/gensim/test/test_data/lee_background.cor'
train_corpus=list()
with open(lee_train_file) as f:
for i, line in enumerate(f):
train_corpus.append(gensim.models.doc2vec.TaggedDocument(gensim.utils.simple_preprocess(line), [i]))
model = gensim.models.doc2vec.Doc2Vec(vector_size=48, min_count=2, epochs=40)
model.build_vocab(train_corpus)
model.wv.vocab['penalty'].count
model.train(train_corpus, total_examples=model.corpus_count, epochs=model.epochs)
line="""
dummy text here...
"""
inferred_vector=model.infer_vector(gensim.utils.simple_preprocess(line) )
model.docvecs.most_similar(inferred_vector, topn=3)
I tried this with list(inferred_vector) but still getting an error.
TypeError: 'numpy.float32' object is not iterable
I am trying to compare the dummy text with the corpus and find if the entry already exist in the data file.
Update:
Instead of list(inferred_vector) I need to use [inferred_vector]. This has solved my problem. But ever-time I run this code, I get different similar documents. How is this possible?
line="""
The national executive of the strife-torn Democrats last night appointed little-known West Australian senator Brian Greig
as interim leader--a shock move likely to provoke further conflict between the party's senators and its organisation.
In a move to reassert control over the party's seven senators, the national executive last night rejected Aden Ridgeway's
bid to become interim leader, in favour of Senator John, a supporter of deposed leader Natasha Stott Despoja and an outspoken
gay rights activist.
"""
inferred_vector=model.infer_vector(gensim.utils.simple_preprocess(line))
model.docvecs.most_similar([inferred_vector], topn=5)
Sometimes I get this list and the list keeps changing everytime I run the code even if there is no change in the model.
[(151, 0.5980586409568787),
(74, 0.5736572742462158),
(106, 0.5714541077613831),
(249, 0.5695925951004028),
(209, 0.5642371773719788)]
[(249, 0.5727256536483765),
(151, 0.5725511312484741),
(74, 0.5711895823478699),
(106, 0.5583171248435974),
(292, 0.5491517782211304)]
As a matter of fact, the first line in training corpus is 99% similar to this line because only 1 word is changed. Surprisingly the document_id 1 is nowhere in the top 5 list.
The dummy line should be selected from lee_background.cor and not from lee.cor
The model text will match with training corpus and not with test corpus.
I have highlighted specific activities (feeding,resting and sleeping) from the dataset in my plot. Now I want to connect these highlighted points in sequence over my polar coordinates.
Here's my dataset:
Activity Latitude Longitude
Feeding 21.09542 71.06014
Resting 21.09564 71.06064
Sleeping 21.09619 71.06128
Walking 21.09636 71.06242
Walking 21.09667 71.06564
Resting 21.09483 71.06619
Can you help me out in this?
# Example dataframe
set.seed(1)
mydf=data.frame(Activity=sample(c("Walking","Feeding","Resting","Sleeping"),20,T),Latitude=rnorm(20,21,0.5),Longitude=rnorm(20,71,0.5))
mydf$Order=1:nrow(mydf)
If you want to connect the points in order regardless of the activity, do the following (for clarity, I added the variable mydf$Order to label the points).
# Plot
library(ggplot2)
ggplot(data=mydf)+
geom_point(aes(x=Latitude,y=Longitude,colour=Activity))+
geom_path(aes(x=Latitude,y=Longitude))+
geom_text(aes(x=Latitude,y=Longitude,label=Order))+
coord_polar(theta="y")
If you want to connect points according to activities, consider CMichael's answer.
Ok I am starting from scratch: My original answerwas much too bulky and inflexible.
Just add the following to get Paths for each Activity without filtering.
+ geom_path(aes(colour=ACTIVITY,x=Latitude,y=Longitude))
If you want to plot only selected Activities:
+ geom_path(data=Data[Data$ACTIVITY %in% c("Sleeping","Resting"),],aes(colour=ACTIVITY,x=Latitude,y=Longitude))
The selected Activities are to be listed in the c(...) vector with each name quoted.
UPDATE: OP clarified that he wants to connect any stationary point, this achieved by running the following:
+ geom_path(data=Data[Data$ACTIVITY!="Walking",],colour="red",aes(x=Latitude,y=Longitude))
Note that the colour=ACTIVITY is removed from the aesthetics and we consider all stationary points (!="Walking") to draw the path.
Code combining the two answers:
set.seed(1)
mydf=data.frame(Activity=sample(c("Walking","Walking","Walking","Walking","Walking","Resting","Feeding","Sleeping"),20,T),Latitude=rnorm(20,21,0.5),Longitude=rnorm(20,71,0.5))
mydf$Order=1:nrow(mydf)
# Plot
library(ggplot2)
ggplot(data=mydf)+
geom_point(aes(x=Latitude,y=Longitude,colour=Activity),size=5)+
geom_path(aes(x=Latitude,y=Longitude),size=1.2)+
geom_text(aes(x=Latitude,y=Longitude,label=Order))+
geom_path(data=mydf[mydf$Activity!="Walking",],colour="red",aes(x=Latitude,y=Longitude)) +
coord_polar(theta="y")
can you suggest a good option for background subtraction using emgucv? my project is real time pedestrian detection.
Not sure if you still need this, but...in EmguCV, if you have 2 images of say type Image<Bgr, Byte> or any other type, called img1 and img2, doing img1 - img2 does work! There is a function called AbsDiff as well, I think it works like this: img1.AbsDiff(img2), you could look into that.
If you already have the picture of the background (img1) and you have the current frame (img2), you could do the above.
This is quite possible take a look at the "MotionDetection" example provided with EMGU this should get you started.
Effectively the code that removes the foreground is effectively named "_forgroundDetector" it is the "_motionHistory" that presents stores what movement has occurred.
The example has everything you need if you have trouble running it let me know,
Cheer
Chris
See:Removing background from _capture.QueryFrame()
This question kind of starts where this question ends up. MATLAB has a powerful and flexible image display system which lets you use the imshow and plot commands to display complex images and then save the result. For example:
im = imread('image.tif');
f = figure, imshow(im, 'Border', 'tight');
rectangle('Position', [100, 100, 10, 10]);
print(f, '-r80', '-dtiff', 'image2.tif');
This works great.
The problem is that if you are doing a lot of image processing, it starts to be real drag to show every image you create - you mostly want to just save them. I know I could start directly writing to an image and then saving the result. But using plot/rectangle/imshow is so easy, so I'm hoping there is a command that can let me call plot, imshow etc, not display the results and then save what would have been displayed. Anyone know any quick solutions for this?
Alternatively, a quick way to put a spline onto a bitmap might work...
When you create the figure you set the Visibile property to Off.
f = figure('visible','off')
Which in your case would be
im = imread('image.tif');
f = figure('visible','off'), imshow(im, 'Border', 'tight');
rectangle('Position', [100, 100, 10, 10]);
print(f, '-r80', '-dtiff', 'image2.tif');
And if you want to view it again you can do
set(f,'visible','on')
The simple answer to your question is given by Bessi and Mr Fooz: set the 'Visible' setting for the figure to 'off'. Although it's very easy to use commands like IMSHOW and PRINT to generate figures, I'll summarize why I think it's not necessarily the best option:
As illustrated by Mr Fooz's answer, there are many other factors that come into play when trying to save figures as images. The type of output you get is going to be dependent on many figure and axes settings, thus increasing the likelihood that you will not get the output you want. This could be especially problematic if you have your figures set to be invisible, since you won't notice some discrepancy that could be caused by a change in a default setting for the figure or axes. In short, your output becomes highly sensitive to a number of settings that you would then have to add to your code to control your output, as Mr Fooz's example shows.
Even if you're not viewing the figures as they are made, you're still probably making MATLAB do more work than is really necessary. Graphics objects are still created, even if they are not rendered. If speed is a concern, generating images from figures doesn't seem like the ideal solution.
My suggestion is to actually modify the image data directly and save it using IMWRITE. It may not be as easy as using IMSHOW and other plotting solutions, but I think it is more efficient and gives more robust and consistent results that are not as sensitive to various plot settings. For the example you give, I believe the alternative code for creating a black rectangle would look something like this:
im = imread('image.tif');
[r,c,d] = size(im);
x0 = 100;
y0 = 100;
w = 10;
h = 10;
x = [x0:x0+w x0*ones(1,h+1) x0:x0+w (x0+w)*ones(1,h+1)];
y = [y0*ones(1,w+1) y0:y0+h (y0+h)*ones(1,w+1) y0:y0+h];
index = sub2ind([r c],y,x);
im(index) = 0;
im(index+r*c) = 0;
im(index+2*r*c) = 0;
imwrite(im,'image2.tif');
I'm expanding on Bessi's solution here a bit. I've found that it's very helpful to know how to have the image take up the whole figure and to be able to tightly control the output image size.
% prevent the figure window from appearing at all
f = figure('visible','off');
% alternative way of hiding an existing figure
set(f, 'visible','off'); % can use the GCF function instead
% If you start getting odd error messages or blank images,
% add in a DRAWNOW call. Sometimes it helps fix rendering
% bugs, especially in long-running scripts on Linux.
%drawnow;
% optional: have the axes take up the whole figure
subplot('position', [0 0 1 1]);
% show the image and rectangle
im = imread('peppers.png');
imshow(im, 'border','tight');
rectangle('Position', [100, 100, 10, 10]);
% Save the image, controlling exactly the output
% image size (in this case, making it equal to
% the input's).
[H,W,D] = size(im);
dpi = 100;
set(f, 'paperposition', [0 0 W/dpi H/dpi]);
set(f, 'papersize', [W/dpi H/dpi]);
print(f, sprintf('-r%d',dpi), '-dtiff', 'image2.tif');
If you'd like to render the figure to a matrix, type "help #avifile/addframe", then extract the subfunction called "getFrameForFigure". It's a Mathworks-supplied function that uses some (currently) undocumented ways of extracting data from figure.
Here is a completely different answer:
If you want an image file out, why not just save the image instead of the entire figure?
im = magic(10)
imwrite(im/max(im(:)),'magic.jpg')
Then prove that it worked.
imshow('magic.jpg')
This can be done for indexed and RGB also for different output formats.
You could use -noFigureWindows to disable all figures.