I am looking for an open source flat file (comma separated values for example) of the major cities in this world, together with their latitude longitudes and timezone. The first 1000 or more is fine. Is such a resource available?
Eventually, a combination of two flats files (lat/long & timezone) is fine too.
You can take a look at this
http://www.partow.net/miscellaneous/airportdatabase/#Download
tells you the lat-long info of all major cities with airports
Have a look at: http://www.maxmind.com/app/worldcities
here is also a free version: http://www.maxmind.com/app/geolitecity
Building on Parapura's answer (+1 for the great idea of using airport lists):
Here is another much more complete airport database (46325 airports as of today and apparently actively maintained).
To hopefully save other's time, here are the few command lines I used to get it all:
wget ourairports.com/data/airports.csv
wget ourairports.com/data/airport-frequencies.csv
wget ourairports.com/data/runways.csv
wget ourairports.com/data/navaids.csv
wget ourairports.com/data/countries.csv
wget ourairports.com/data/regions.csv
To get major cities, I would use the airports qualified as "large_airport".
I had this problem - I needed it to place markers on a map. Here is the procedure that I followed:
https://gist.github.com/rolfen/cdaf1895b44659c1c4d2b90970af443e
The data is provided by Wikipedia:
https://en.wikipedia.org/w/index.php?title=List_of_cities_by_longitude&printable=yes
I do some processing on that using the browser debugging console.
The output is a JSON object. If you can work with JavaScript, it should be easy to tweak.
This one ships as CSV files http://www.geobytes.com/geoworldmap/ and contians latitude longitudes and timezones.
I've found one that's still on the web and kept reasonably up-to-date: https://simplemaps.com/data/world-cities
The Basic version (approx. 41 thousand locations) is available free under Creative Commons licensing in CSV or Excel formats. Lat, long, and population are all included. I've just downloaded it, so no quality testing yet.
I would recommend NaturalEarth where you can download files with major cities including their names in different languages and their population. Country borders and time zones are provided as well. The download formats include SHP, SQLite and GeoPackage; no CSV though, but all are easy to read in QGIS (where you can convert them to CSV) or e.g. Python.
If you need smaller cities as well have a look at https://www.geonames.org/ - at that site CSV downloads of cities are provided (with coordinates, names and other properties).
Related
I am not a native english speaker so please ignore my mistakes.
I want to download complete map (highway,ways) of Pakistan and complete nodes from overpass-turbo in geojson format.
I already downloaded one city data but when I went to download complete country map, it gave me a error.
Here how I download one city.
1.I select a area form overpass-turbo and select highway and type = way just before run a query which is shown below
When I click on build and run query, after a while it looks like
3.So Data is ready for export, when I click on export it gives me option to download data as a geojson extension type which is..
I downloaded and used my data,
NOW problem is that when I go to download complete country data it gives me error which is
I also increases query time but I don't know what is the problem.
Please help me to solve this problem.
Link where I am downloading map
https://overpass-turbo.eu/
Overpass API is the wrong source for downloading complete countries. Overpass API can be used for downloading small data extracts. All highways of a specific country are probably just too much.
As an alternative use a country or area extract. Afterwards filter the data for specific tags by using the tags-filter option from osmium-tool. Alternative software for filtering are osmosis and osmfilter.
The following osmium command should extract all ways with a highway tag from the raw data you downloaded:
osmium tags-filter -o highways.osm.pbf pakistan.osm.pbf w/highway
Project Environment
The environment we are currently developing is using Windows 10. nodejs 10.16.0, express web framework. The actual environment being deployed is the Linux Ubuntu server and the rest is the same.
What technology do you want to implement?
The technology that I want to implement is the information that I entered when I joined the membership. For example, I want to automatically put it in the input text box using my name, age, address, phone number, etc. so that the user only needs to fill in the remaining information in the PDF. (PDF is on some of the webpages.)
If all the information is entered, the PDF is saved and the document is sent to another vendor, which is the end.
Current Problems
We looked at about four days for PDFs, and we tried to create PDFs when we implemented the outline, structure, and code, just like it was on this site at https://web.archive.org/web/20141010035745/http://gnupdf.org/Introduction_to_PDF
However, most PDFs seem to be compressed into flatDecode rather than this simple. So I also looked at Data extraction from /Filter /FlateDecode PDF stream in PHP and tried to decompress it using QPDF.
Unzip it for now.Well, I thought it would be easy to find out the difference compared to the PDF without Kim after putting it in the first name.
However, there is too much difference even though only three characters are added... And the PDF structure itself is more difficult and complex to proceed with.
Note : https://www.adobe.com/content/dam/acom/en/devnet/pdf/pdfs/PDF32000_2008.pdf (PDF official document in English)
Is there a way to solve the problem now?
It sounds like you want to create a PDF from scratch and possibly extract data from it and you are finding this a more difficult prospect than you first imagined.
Check out my answer here on why PDF creation and reading is non-trivial and why you should reach for a tool you help you do this:
https://stackoverflow.com/a/53357682/1669243
I'm new to this forum and to Orange.
I don't really now Python at this point but am ready to learn.
However, before going further in this environment I would like to know if it can answer my needs !
What I am basically doing is "transforming" PDF product catalogues into Excel files that can be used by another software to create a database for another software.
I have tiles catalogues in PDF just like this one :
and turn it into this type of xls table : http://imgur.com/BtLBkOS
I basically need it to retrieve the article number, the colour, the size (e.g: 20x20). The G/B parts are completed manually after it has been done.
All catalogues are not the same so I sorted out some using pdftotext, RegEx with Notepad++
But I would like to know if this data mining solution could work it out ?
Orange does not support reading PDF files. You will have to use specialized utilities or program it yourself.
I am working on a game development project with Unity3D using TortoiseHg to track our files. We are nearing the end of development and we would like to prepare a deliverable that displays our development data in a nice info-graphic, like this image below
I am aware of how to create the charts, I am just having trouble understanding how to export the data in an easy to read format from TortoiseHg. I would like to export all this data if possible:
If someone could assist me in educating me on a method of exporting the above data into a spreadsheet I would appreciate it.
Thanks,
-Naeem
Well, in any case you have to use CLI version on Mercurial (hg in command prompt, with always exist behind the scene of TortoiseHG)
If you see at log pane, you have to use hg log and include in output all needed for you data and format it in way, usable for importing into spreadsheet (CSV isn't bad choice)
In order to write full log command, you have to know, which details about commit you have to have in data-file as result (forhet about format, think about content)
AFAICS from graph, for each commit you have to know
Month of commit
Day of month
Time (local or GMT?)
Is it the full list of requirements?
I'm wondering if there's an easy way to download a large number of files of one arbitrary type, e.g., downloading 10,000 XML files. In the past, I've used Bing's API. It's free and offers unlimited queries. However, it doesn't index as many types of files as Google does. Google indexes XML files, CSV files, and KML files. (These can all be found by doing searches like "filetype:XML".) As far as I know, Bing doesn't index these in a way that's easily searchable. Is there another API that has these capabilities?
How about using wget? You can give wget a URL (for example, a google search result) and tell it to follow all the links on that page and download them (I bet you could also give it a filter).
Just tried it and got an ERROR 403: Forbidden. Apparently Google blocks requests from Wget. You'll have to provide a different user agent. Quick search provided this example:
http://www.mail-archive.com/wget#sunsite.dk/msg06564.html
Then it worked with the example given.