My intentions:
Actually, I intend to:
implement vehicles as containers
simulate/move these containers on the .osm maps-based roads
My viewpoint about the problem:
I have loaded the XML-based .osm file and processed it in python using xml.dom. But I am not satisfied with the performance of loading the .osm file because later on, I will have to add/create more vehicles as containers that will be simulated onto the same road.
Suggestions needed:
This is my first time to solve a problem related to maps. In fact, I need suggestions on how to proceed by keeping in mind, the performance/efficiency, with this set of requirements. Suggestions in terms of implementation will be much appreciated. Thanks in advance!
Simulating lots of vehicles by running lots of docker containers in parallel might work I suppose. Maybeyou're initialising the same image with different start locations etc passed in as ENV vars? As a practical way of doing agent simulations this sounds a bit over-engineered to me, but as an interesting docker experiment it might make sense.
Maybe you'll need a central thing for holding and sharing the state (positions of other vehicles) and serving that back to the multiple agents.
The challenge of loading an .osm file into some sort of database or internal map representation doesn't seem like the hardest part, and because it may be done once on initialisation and imagine it's not the most performance critical part of this.
I'm thinking you'll probably want to do "routing" through the road network (taking account of one ways etc?), giving your agents a purposeful path to follow to a destination. This will get more complicated if you want to model interactions with other agents e.g. you might want to model getting stuck in traffic because other agents are going the same way, and even decisions to re-route because of traffic, so you may want quite a flexible routing system, perhaps self-coded.
But there's lots of open source routing systems which work with OSM data, to at least draw inspiration from. See this list: https://wiki.openstreetmap.org/wiki/Routing#Developers
Popular choices like OSRM are designed to scale up to country size or even global openstreetmap data, but I imagine that's overkill for you (you're probably looking at simulating within a city road network?). Even so. Probably easy enough to get working in a docker container.
Or you might find something lightweight like the code of the JOSM routing plugin easier to embed in your docker image and customize (although I see that's using a library called "JGraphT")
Then working backwards from a calculated route you can calculate interpolated steps along that path which will allow you to make your simulated agents take a step on each iteration (simulated movement)
Related
I need to do some basic networking for a Pygame project.
Basically, it's a 2D single player or cooperative game. The networking only needs to support two players, with one as a host.
The only information that needs to be sent is the positions of players, creeps and bullets.
I've been reading around and Twisted keeps coming up, but I haven't done networking before, and I'm not sure if that might be an overkill.
So, is it possible for a relative newbie to implement networking in Pygame?
This was asked recently on Reddit, so I'll more or less just copy my answer over from there. I apologize for not being able to provide more links, I have <10 rep so I can only post two at a time.
Twisted might work, but I don't have a whole lot of experience with it. I'd recommend going with sockets, as that's what Twisted uses in the background anyway. Beej's guide (google it) is pretty much the Holy Bible of sockets if you want to learn how they work (in C++, but the concepts extend everywhere). Python does abstract some of the complexity away, but it's still a good idea to know what's going on in the background.
For Python specific sockets, you can go ahead and just use the howto (user745294 posted a link above). Here's a nice article titled "What every programmer needs to know about Game Networking". It goes into the different types of major networking styles (client-server, p2p, udp v. tcp, etc.) and the history behind what some major games used for their networking.
Below is a link to a demo I did on making a networked "game" in Python 2.6/Pygame. It's not actually a game, but each client you create connects to the server and controls a character. You can move your character with the arrow keys and the character will move on all connected clients. I tried commenting the source code with some indication of what I'm sending back and forth, but you may need a little knowledge about sockets to understand it.
The source code is provided in the codepad links in the comment below this post. You will need to provide two images in the same directory as the scripts:
bg.png is the background sprite. It should be an image 400px wide and 300px tall (this can be changed in the GameClient class if needed)
sprite.png is the player character. It should be smaller than the background so that you can see it moving around.
You can use Twisted for networking with Pygame. The "game" project on Launchpad has some examples of how one might integrate the main loops together; basically, use twisted.internet.task.LoopingCall to draw Pygame frames and handle input, while letting the Twisted reactor of your choice run normally.
Since you are already using Pygame, I think this light networking library made for Pygame will do what you need and teach you, but not overwhelm you.
"Mastermind Networking Lib" via pygame.org
There is Pyro (Python remote objects) as another solution for networking in Python.
http://irmen.home.xs4all.nl/pyro/
Using raw sockets is low-level and full of danger. As said before, Twisted is complex and takes to time get up and running. To save yourself some headaches I'd try something like zerorpc.
You need the following solutions:
discovering other player(s) on the (local) network, you don't want player to enter some IP address
handle network errors
serialize messages containing your data (positions, player name, etc.)
handle threading as networking is asynchronous I/O
Above should still be called 'basic', you should really use some fancy networking library with idiomatic API.
Essentially you need to expose the network service (in its own thread) that will push messages to Python's Queue, and then access this same queue from your Pygame code, and if there is a message then you update whatever structures you use to store player's position and draw it on screen.
You shouldn't send stuff like bullet positions over the network as they can be easily (and faster) calculated locally. You just send an event like bullet_shot over the network with a source position and velocity vector.
I have Geoserver 2.11.2,PostgreSQL 9.5,open layers 3 and Tomcat 8 all are installed on Ubuntu 16.4 Azure cloud virtual machine.I also enabled GeoWebCache but still WMS layer rendering speed is slow(15 to 16 seconds).Please find this .Is there any idea to improve more speed than the current speed of web-tool,Thanks.
Broadly, it sounds like something is misconfigured. There are some excellent resources in the GeoServer docs (http://docs.geoserver.org/stable/en/user/production/) about running in production. From GeoSolutions, there is some training materials (http://geoserver.geo-solutions.it/edu/en/enterprise/index.html) and talks (https://www.slideshare.net/geosolutions/geoserver-in-production-we-do-it-here-is-how-foss4g-2016) which address common techniques for data prep, JVM options, and other considerations which may help some.
As a particular call-out, I'd strongly suggest Marlin (https://github.com/bourgesl/marlin-renderer/wiki/How-to-use). Its use in GeoServer can help immensely with concurrent rendering (http://www.geo-solutions.it/blog/developerss-corner-achieving-extreme-geoserver-scalability-with-the-new-marlin-vector-rasterizer/).
It may be worth making sure that PostGIS is installed and that your data has a spatial index. Tuning PostGIS is a separate topic.
Once the data is prepped and indexed and Marlin is up and running, it may be worth seeding the GWC cache. With that, your application would just be serving pre-baked tiles for coarse zoom levels and that should be snappier.
Looks like you have a lot of layers turned on in your map. Just zooming in once triggered a total of 700 individual tile requests, most of them to your GeoServer. I don't think your main problem is your GeoServer (although tweaking it using the other answers suggestions is always a good idea), I think your main problem is simply throughput.
Most browsers have a limit (when using http 1.1) of how many simultaneous requests can be sent to the same domain, once you hit that limit, all other requests are queued until the previous ones are done. I think that's your problem, your server is dealing with the requests as quickly as it can, but there are so many that it simply cannot serve them at the speed you are expecting.
I would strongly recommend you look at reducing the number of layers you have loaded by default, or implement some kind of zoom restriction so that certain layers turn off at different zoom levels. You could even think about combining a number of the layers into one and perhaps using GeoServers CQL filtering to change what is displayed.
I am currently using josdirkson's SVG extrude script to form groups of 20-30 complex shapes. My goal is to individually manipulate each object as well as the group as a whole throughout the user's interaction. I have been able to achieve this so far, however, my load time can range from 7 to 20 seconds on a variety of devices. I was wondering if a lot of this could be just inherent in script that converts all the SVG paths into bezierCurves, etc. If this was the case, I was wondering if a viable solution might be to somehow export from Three.js to a JSON or other file type which would then be the subsequent data source users are loading from. I was looking at this thread briefly, but didn't want to get too far ahead of myself before crowd sourcing a solution! Any advice or input is greatly appreciated! Thank you!
Best advice I can give you is to have a look at the profile of the loading-process. To test this in chrome, you can add pairs of console.profile('something'); and console.profileEnd('something'); calls to your code for the region you want to analyze. Then open up the Profiles-panel in the devtools and reload the page / re-run the javascript.
This will probably be able to tell you if you are right about your assumptions. At least it will help you find the thing the time in JS is spent on.
And if that's really the case, you could do some caching of geometries, using geometry.toJSON() and new THREE.JSONLoader().parse(json) to save and restore the geometries. In most cases this should be significantly faster than somehow computing the geometry. (note: there are other, more space-efficient and even more performant ways to do the caching, but the json-format is a good place to start)
This is one of my assignments and I need some help in getting started. The basic idea behind the assignment is that I have to design a self destructible email program that is capable of destructing the message after (n) time duration.
Speaking about self destructible emails, there are quite a few ones on the internet offering the same service. But what they do is, they just convert the email message into an image and store them on their servers. Now, they send the message attaching the image inline with it. After they receive a hit on that image (which means that the message was being opened), they simply delete the image and the inline image link breaks! BOOM!
IMO, that's not what a self destructing email should be like. Nevertheless, in my case, I have to take care of following points:
I have to do it for TEXT. No image, nothing else.
I have to assume that the systems used throughout the process will be UNIX based (I don't know how that is going to make a difference).
There are also some hints regarding the usage of various network layers in solving the problem.
This isn't supposed to be done "in general". What I mean by that is, I have to do that ONLY for one/two UNIX systems. Let me put it this way, all I have is two UNIX systems and nothing else. Now I want to create a program (in UNIX itself) that would do that self-destructing thing. I have total control of protocols and the network layers and I have to code anything and everything required at any level.
This is more geared towards the StackOverflow side of things but I have no problem getting you started.
The first thing I'd like to point out is that you seem to be heavily over-analyzing this. The services that have self-destructing e-mails which are image based are simply deleting a file after it is viewed. All you need to do differently is put that text in a file and get it's contents before deleting it. This fits well with the UNIX philosophy since so many programs already make use of flat files.
The part you seem to have left out is how you are building this. You describe it as an e-mail program and then talk about web services. Is this a web-based project or a program you are designing for Linux? Do you have to code everything from scratch or can you parse output from Linux utilities to grab the mail? These kinds of things really would simplify the process.
I want to code a trading bot for Magic: The Gathering Online. This bot should wait until someone offers to trade, accept, look through the cards available from the other trader (the information is shown on screen), and perform other similar functions. I have several questions:
How can it know that someone is offering a trade?
How can it know that the other trader has some card (the informaion is stored in pictures)?
I just cannot imagine right now how to do it, I have no experience with it, until now I've been coding only console programs for my physics neŃessities.
First, you should note that some online games forbid bots, as they can give certain players unfair advantages. The MTGO Terms of Service do not seem to say anything about this, though they do put restrictions on anything that might negatively impact the service. They have also said that there is a possibility they will add an API in the future, so they don't seem to be against the idea of automation, but are not supporting it at the moment. Tread carefully here, but it looks like it should be OK to write a bot as long as it is not harmful or abusive. This is not legal advice, and it would be a good idea to ask the folks who run MTGO for permission. edit since I wrote this, it has been pointed out that there are lots of bots already, so there should be no problems writing bots.
Assuming that it is not forbidden by the terms of service, but they do not have an API, you will have to find a way to detect what's going on, and control the game automatically. There's a pretty good series of articles on writing poker bots (archived copy), which has some good information on how to inject a DLL into an application, scrape the screen, and control the application. That might provide you with a starting point for doing this sort of thing.
You might also want to look for tools that other people have already written for doing this. It looks like there are several existing MTGO bots, but they all seem a bit sketchy (there have been some reports of them stealing passwords), so be careful there.
Edit
Since this answer still seems to be getting upvotes, I should probably update it with some more useful information. Since writing this, I have found a great UI automation system called Sikuli. It allows you to write programs in Python that automate a GUI. It includes image recognition features which make it very easy to recognize buttons, cards, and other UI elements; you just take a screenshot, crop it down to include just the thing you're interested in, and do fuzzy image matching (so that changing backgrounds and the like doesn't cause the match to fail). It even includes a custom IDE that allows you to embed those screenshots directly in your source code, so you can see exactly what the code is looking for. Here's an example from the documentation (apologies for the code formatting, doing images inline in code is not easy given StackOverflow's restricted subset of HTML):
def resizeApp(app, dx, dy):
switchApp(app)
corner = find(Pattern().targetOffset(3,14))
drop_point = corner.getTarget().offset(dx, dy)
dragDrop(corner, drop_point)
resizeApp("Safari", 50, 50)
This is much easier to get started with than the techniques mentioned in the article linked above, of injecting a DLL into the process you are debugging. Sikuli runs entirely at the UI level, so you never have to modify the program you are automating or worry about changes to the internals breaking your script.
One thing it is a bit poor at is handling text; it has OCR features, but they aren't all that good. If the text is selectable, however, you can select the text, copy it, and then look directly at the clipboard.
If I were to write a bot to automate something without a good API or text-based interface, Sikuli is probably the first tool I would reach for.
This answer is constructed from my comments.
What you are trying to do is hard, any way you try and do it.
Arguably the easiest way to do it is to totally mimic the user. So the application presses buttons, moves the mouse etc. The downside with this is that it is dependant on being able to recognise the screen.
This is easier if you can alter the games files as you can then just skin ( changing the image (texture)) the required cards to a single unique colour.
The major down side is you have to have the game as the top level window or have the game running in a virtual machine. Neither of which is ideal.
Another method is to read the processes memory. You may be able to find a list of memory locations, which would make things simpler, otherwise it involves a lot of hardwork, a debugger to deduce the memory addresses. It also helps (a lot) to be able to understand assembly.
The third method is to intercept the packets, and alter them. This is easier that the method above as it (at least for me) is easier to reverse engine the protocol as you have less information to deal with. It is just a matter of setting up a packet sniffer and preforming a action with one variable different (for example, the card) and comparing the differences.
The thing you need to check are that you are not breaking the EULA. I don't know how the game works, but most of the games I have come across have a EULA that prohibits (i.e. You get banned) doing any of the things I have mentioned.