I'm attempting to mass convert the Sinusoidal projections of hundreds of MODIS single-band files to WGS84 in a single directory (which were extracted from HDF-EOS format using gdal_translate) using GDAL in Cygwin.
When I execute the gdalwarp command on a single file, it executes the command successfully and the result is my desired output:
$ gdalwarp -s_srs "+proj=sinu +R=6371007.181 +nadgrids=#null +wktext" -t_srs EPSG:4326 testfile.tif testfile_proj_1.tif
Processing testfile.tif [1/1] : 0Using internal nodata values (e.g. -1) for imag
e testfile.tif.
...10...20...30...40...50...60...70...80...90...100 - done.
However, when I try to process the aforementioned hundreds of files using a for loop...:
$ for f in *.tif; do
> gdalwarp -s_srs "+proj=sinu +R=6371007.181 +nadgrids=#null +wktext" -t_srs EPSG:4326 "$f" "${f%.*}.tif"
..., it returns errors:
ERROR 1: Too many points (529 out of 529) failed to transform, unable to compute
output bounds.
Warning 1: Unable to compute source region for output window 0,0,2400,2400, skip
ping.
I am baffled as to why the command will work on a single file, but won't in a mass conversion.
How do I get this to work? Are there any alternatives to GDAL that I can use to mass reproject TIFFs if this is a wall I hit?
Thank you in advance.
Related
I have tiled png files and those projection is EPSG:4326.
I convert projection to EPSG:3857 with below 2 commands:
gdal_translate -of Gtiff -a_ullr 135.00000000000003 36.59788913307022 140.62500000000003 31.952162238024975 -a_srs EPSG:4326 4326.png 4326.tiff
gdalwarp -s_srs EPSG:4326 -t_srs EPSG:3857 4326.tiff 3857.png
Can I make it with 1 command ?
I don't know how to do it with one command, but if you are working on a Unix-like environment, you can use a pipe to avoid creating an intermediate file.
gdal_translate -of VRT -a_ullr 135.00000000000003 36.59788913307022 140.62500000000003 31.952162238024975 -a_srs EPSG:4326 4326.png /vsistdout/ | \
gdalwarp -t_srs EPSG:3857 /vsistdin/ 3857.png
/vsistdin/ and /vsistdout/ are part of a feature called "GDAL Virtual File Systems", which represents a virtual file that represents standard input and standard output. The output format of gdal_translate could be GTiff, but I tried to use VRT which contains only metadata. If this doesn't work, try changing VRT to GTiff.
I'm trying to unzip a directory of 75,000 images for training a CNN. When unzipping using,
!unzip -uq "/content/BDD_gt_8045.zip" -d "/content/drive/My Drive/Ground_Truth"
not all images unzip. I have about 5,000 I believe. I tried doing it several times but then I have some duplicates. Is there a limit to the number of images I can unzip?
I'm currently stuck on how else I'm meant to get all files into my drive to train the model.
Colab's default 'unzip' binary doesn't work as expected. It seems to cancel the unzipping automatically after a few cycles. Run latest version of 7z and you are good to go.
# To extract with full paths
!7z x <filename.zip>
# To extract all the files in the same folder (ignore paths)
!7z e <filename.zip>
# To specify output directory, use '-o'
!7z x <filename.zip> -o '/content/drive/My Drive/Datasets/FashionMNIST'
I'm trying to change formants of the audio file with praat in Colab. I found the script that does that, it's code and the code for calculating formants. I installed praat:
!sudo apt-get update -y -qqq --fix-missing && apt-get install -y -qqq praat > /dev/null
!wget -qqq http://www.praatvocaltoolkit.com/downloads/plugin_VocalToolkit.zip
!unzip -qqq /content/plugin_VocalToolkit.zip > /dev/null
with open('/content/script.praat', 'w') as f:
f.write(r"""writeInfoLine: preferencesDirectory$""")
!praat /content/script.praat
/root/.praat-dir
!mv /content/plugin_VocalToolkit/* /root/.praat-dir
!praat --version
Praat 6.0.37 (February 3 2018)
How can I apply this script to multiple wav files without UI, using linux command line or python?
The general answer
You don't. You run a script, and it's entirely up to the script how it works, what objects it works on, where those objects are fetched, how they are fetched, etc.
So you always have to look at how to apply a specific script, and that always entails figuring out how that script wants its input, and how to get to that point.
The specific answer
The page for the script you want says
This command [does something on] each selected Sound
so the first thing will be to open the files you want and select them.
Let's assume you'll be working with a small enough number of sounds to open them all in one go. If you are working on a lot of sound files, or files that are too large to hold in memory, you'll have to batch the job into smaller chunks.
One way to do this would be with a wrapper script that opened your files, selected them, and executed the other script you want:
# Get a list of all your files
files = Create Strings as file list: "list", "/some/path/*.wav"
total_files = Get number of strings
# Open each of them
for i to total_files
selectObject: files
filename$ = Get string: i
sounds[i] = Read from file: "/some/path/" + filename$
endfor
# Clear the selection
nocheck selectObject(undefined)
# Add each sound to your selection
for i to total_files
plusObject: sounds[i]
endfor
# Run your script
runScript: path_to_script$, ...
# where the ... is the list of arguments your script expects
# In your specific case, it would be something like
runScript: preferencesDirectory$ + "/plugin_VocalToolkit/changeformants.praat",
... 500, 1500, 2500, 0, 0, 5500, "yes", "yes"
# ,-´ ,-´ ,--´ ,--´ ,-´ ^ ^ ^
# New F1, F2, F3, F4, and F5 means | | |
# Max formant | |
# Process only voiced parts |
# Retrieve intensity contour
# Do something with whatever the script gives you
My Praat is pretty rusty, but this should at least give you an idea of what to do (disclaimer: I haven't run any of the above, but the concepts should be fine).
With that "wrapper" script stored somewhere, you can then execute it from the command line:
$ praat /path/to/wrapper.praat
I am trying to apply a gmic filters using apply_video option of gmic utility for linux .
./gmic --apply_video 'small.mp4','-denoise 30,10' -o convert.avi
but the output is an error in terminal
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Apply command 'small.mp4,-denoise 30,10' on video stream '', with output filename '', first frame 0, last frame -1 and frame step 1.
[gmic] *** Error in ./apply_video/*local/*substitute/ *** Command '-basename': Undefined argument '$1', in expression '$1' (for 2 arguments specified).
[gmic] Command '-basename' has the following description:
-basename:
file_path,_variable_name_for_folder
Return the basename of a file path, and opt. its folder location.
When specified 'variable_name_for_folder' must starts by an underscore
(global variable accessible from calling function).
so what is the correct syntax for gmic --apply_video ?
I am using version 1.7.3
I'm using latest available version 1.7.5_pre, and I use it like this :
$ gmic -w -apply_video input.avi,\"-denoise 30,10\",0,-1,1,output.avi
For G'MIC options, you can use the excellent offline help
gmic -h
or
gmic -h -apply_video
Which as of version 1.7.8 gives
gmic: GREYC's Magic for Image Computing.
Version 1.7.8, Copyright (c) 2008-2016, David Tschumperle.
(http://gmic.eu)
-apply_video:
video_filename,_"command",_first_frame>=0,_last_frame={ >=0 | -1=last },
_frame_step>=1,_output_filename
Apply a G'MIC command on all frames of the specified input video file, in a streamed way.
If a display window is opened, rendered frames are displayed in it during processing.
The output filename may have extension '.avi' (saved as a video), or any other usual image file
extension (saved as a sequence of images).
Default values: 'first_frame=0', 'last_frame=-1', 'frame_step=1' and
'output_filename=(undefined)'.
I've written a program to process a bunch of png files that are generated by a seperate process. The capture mostly works, however there are times when the process dies and is restarting which leaves a corrupted image. I have no way to detect when the process dies or which file it dies one (there are ~3000 png files).
Is there a good way to check for a corrupted png file?
I know this is a question from 2010, but I think this is a better solution: pngcheck.
Since you're on a Linux system you probably already have Python installed.
An easy way would be to try loading and verifying the files with PIL (Python Imaging Library) (you'd need to install that first).
from PIL import Image
v_image = Image.open(file)
v_image.verify()
(taken verbatim from my own answer in this thread)
A different possible solution would be to slightly change how your processor processes the files: Have it always create a file named temp.png (for example), and then rename it to the "correct" name once it's done. That way, you know if there is a file named temp.png around, then the process got interrupted, whereas if there is no such file, then everything is good.
(A variant naming scheme would be to do what Firefox's downloader does -- append .partial to the real filename to get the temporary name.)
Kind of a hack, but works
If you are running on linux or something like you might have the "convert" command
$ convert --help
Version: ImageMagick 5.5.6 04/01/03 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 2003 ImageMagick Studio LLC
Usage: convert [options ...] file [ [options ...] file ...] [options ...] file
If you make an invalid png, and then try to convert, you'll get an error:
$ date> foo.png
$ convert foo.png foo.gif
convert: NotAPNGImageFile (foo.png).
Find all non-PNG files:
find . -type f -print0 | xargs -0 file --mime | grep -vF image/png
Find all corrupted PNG files:
find . -type f -print0 | xargs -0 -P0 sh -c 'magick identify +ping "$#" > /dev/null' sh
file command only checks magic number. Having the PNG magic number doesn't mean it is a well formed PNG file.
magick identify is a tool from ImageMagick. By default, it only checks headers of the file for better performance. Here we use +ping to disable the feature and make identify read the whole file.