I'm looking for a decentralized database for my Dapp.
I found two helpful solutions orbitdb and gun but I couldn't realize the main difference and which one is more suitable for a social-media-like application that use ethereum smart contract.
I'm the author of gun. I've seen OrbitDB floating around (ha, pardon the pun!) but obviously am not intimately knowledgable about it enough to speak about it. I'll leave that for them to answer. :)
It does look like OrbitDB also uses CRDTs, which means they are probably doing things right/correct. Very few people are using CRDTs, which is sad, but makes GUN pretty unique! Despite the last 4 years I've been trying to evangelize to people that they should use CRDTs, I welcome it.
Maybe a practical way to answer your question, #maroodb , is around GUN's use case for social media:
Yes, GUN is already being used for P2P social media dApps. You can do:
User account systems, including traditional username/password, and password reset (no server, fully P2P).
Published user data, like tweets, etc. that cannot be tampered with by anybody else (signature encryption, but not private).
Private data to the user (cypher encryption).
Secret data between 2 users, like private messages (diffie-hellman).
And we're currently working on making it easier to do group read/write data, which is possible, but no demos yet.
Here are some links that show demos, and other resources, that might be helpful in building your app:
Conceptual understanding of P2P security, done in 1 minute animated cartoon explainers - https://gun.eco/explainers/data/security.html
Short, fast article on how to get started with user accounts for a P2P Twitter - https://hackernoon.com/so-you-want-to-build-a-p2p-twitter-with-e2e-encryption-f90505b2ff8
A 4 minute and 40 line of code interactive coding tutorial, on how to build a super basic P2P Twitter - https://scrimba.com/c/c2gBgt4
Social network idea - https://d.tube/#!/v/marknadal/lanz4e6z (d.tube itself is a P2P youtube)
P2P user account management system - https://d.tube/#!/v/marknadal/gfqglxvd
Super basic Private Message demo in 75 lines of code - https://d.tube/#!/v/marknadal/ganoayt8
Documentation on how to use the security API - https://gun.eco/docs/SEA
And of course, a super friendly chat room of Open Source developers helping each other out build these kind of stuff, https://github.com/amark/gun ! :)
You'll have to ask OrbitDB for their examples/use case for P2P social networking dApps, maybe they know what makes gun different.
Note: A GUN+IPFS adapter will be out soon.
Ethereum is great, but your social network might not scale up if it is implemented primarily as an ethereum smart contract, just as a warning.
We recommend you use Ethereum for any cryptocurrency payments or transactions for your user (not GUN or Orbit).
But we do not recommend building basic social network features (profile info, posts, messages, etc.) using Ethereum.
Likely, 90 to 95% of your app will be GUN or Orbit, and possibly only 3 to 5% smart contracts.
Related
These days I am finding myself in the position of having to implement for one of my college courses a system that should act as a giant wrapper over many communications apps like Gmail , Facebook Messenger maybe even WhatsApp .To put it simply you should have a giant web interface where you can authorize Gmail , Messenger and use them at once when required. I am thinking of going with an REST API to manage the user's services authorized by OAuth2.Also I am thinking of using Node.JS and Express.js in the backend and React.js in the frontend. I found some sweet libraries in npm that should take care of interacting with the involved APIs(https://www.npmjs.com/package/node-gmail-api this one for instance), but I am also doubtful about this approach , for example I have no idea how to keep the use notified about its incoming mails or messages for example . I am in dire need of some expertise since I forgot to mention but I am quite the newbie in this field. To sum it up for once my question is how would you implement such an infrastructure ? Is it my approach viable or I am bound to hit some really hard to overcome obstacles?
As a college exercise, it would be a really fun experiment, so it definitely worth the time you want to put into it. However, once you want to add more features, the complexity will go up pretty fast.
Here are a couple of ideas you can think of:
It's pretty clear that your system can't do more things than the capabilities exposed by the APIs of communication apps (e.g. you can't have notifications in gmail if the API doesn't have this capability).
For that reason, you should carefully study the APIs and what functionalities they expose. They have public docs that you can check out: (Gmail API, Facebook Messanger API)
Some of the apps you want to communicate with may not have an official API (e.g. WhatsApp) - those kinds of details you definitely want to know from the start.
Based on the analysis of those APIs, you should lay out a list of requirements for your system, which can be extracted from all the APIs, for example: message notifications, file transfers, user profiles, etc.
In this way, you know exactly what capabilities your system should have, and you don't end up implementing a feature that is available only in 1 API out of 4.
Also, it would be a bit challenging to design your system from a user perspective, because the apps have different usage patterns - chat apps, where messages are coming in real-time, vs email, which is not real-time communication. That's just a detail anyway, the gist of your project is to play with those APIs.
Also, it may worth checking out the Gateway Aggergation Pattern, which is related to this project - you may want the user to send a message to multiple apps, by using a single request to your service.
I have an e-commerce website and I need to display products in homepage based on user interest like advertisement showing on facebook and google based on the search we did on the internet.
Is there any API from facebook or google or any website for fetching user interests?
I have wondered how facebook, google, and booking.com collaboratively tracking us even if they are different companies.
Is there any common gateway or common interface where these big companies share user info by making a system with cookies to track among all as a centralized big data model for behavioral advertising?
I am looking for answers from experts for this. I really need to build a system for tracking user interest based on their search on google and other websites.
So you need to use something called recommender systems, using a machine learning algorithm, you'll recommend products to people based on ratings and interest. by using previous data of the same user or other users (just like when you get recommended videos on youtube).
this topic is too big for me to explain it step by step in here, and you need to first have a good understanding of primitive machine learning such as classification, regression ... etc
so if you're interested make sure to check a coursera course called Machine Learning (Stanford University) it's taught by machine learning rockstar Andrew NG, and it doesn't only teach you machine learning, but it takes you from somebody with no idea on the subject to an expert (technically) the course used MATLAB/OCTAVE and it has an entire section on Recommender Systems which is what you need, after you've finished just implement what you learned in the language of your choice !
PS:
you can always look up tutorial online for implementing Recommender Systems, but you will waste so much time because you would have no idea about what you're doing without understanding the theory which you will master in the course I've recommended, the course can also be found easily on youtube. but taking it for FREE in coursera will help you more because you'll have hands-on programming experience, on the different subjects.
For that google and facebook has their algorithm to track user movement and they use it for showing ads on their website.
i don't think that is available for common use.
I think you won't be able to Track there interest live Facebook, Google, Amazon, twitter. They are collecting your Interest form there own platform.
Also they manage large Add provider. SO once a Add has been clicked by you, it has been tracked. Also Google Play, itunes has access to your Phone.
I want to develop a simple website that will have simple features: user management, online payment and some simple data management. On top, I want to develop an iPhone app that will interact with the users's data. And in a close future, I would develop an Androïd app.
Now I need to develop this application in a very secure way, from user authentication to data security.
I know a little of web frameworks like Symfony 1 and Django. And I'm a good Java programmer.
What would be your choice of technology to achieve my to goals of security and easy iApp interaction?
Thanks
I would learn Symfony 2. They took great efforts to improve the security for 2, including getting a security assessment for the core framework and doctrine. There are a lot of changes from sf1.x, but the jump isn't too bad (i just did it myself).
Symfony is a good base for a new project, but it wont make everything you do secure. That is far beyond the scope of a post on stackoverflow. My best advice: read some of the articles on http://owasp.org, maybe watch a few presentations on specific subjects you are worried about, and keep an eye on security blogs/mailing lists.
While the are many social networks in the wild, most rely on data stored on a central site owned by a third party.
I'd like to build a solution, where data remains local on member's systems. Think of the project as an address book, which automagically updates contact's data as soon a a contact changes its coordinates. This base idea might get extended later on...
Updates will be transferred using public/private key cryptography using a central host. The sole role of the host is to be a store and forward intermediate. Private keys remain private on each member's system.
If two client are both online and a p2p connection could be established, the clients could transfer data telegrams without the central host.
Thus, sender and receiver will be the only parties which are able create authentic messages.
Questions:
Do exist certain protocols which I should adopt?
Are there any security concerns I should keep in mind?
Do exist certain services which should be integrated or used somehow?
More technically:
Use e.g. Amazon or Google provided services?
Or better use a raw web-server? If yes: Why?
Which algorithm and key length should be used?
UPDATE-1
I googled my own question title and found this academic project developed 2008/09: http://www.lifesocial.org/.
The solution you are describing sounds remarkably like email, with encrypted messages as the payload, and an application rather than a human being creating the messages.
It doesn't really sound like "p2p" - in most P2P protocols, the only requirement for central servers is discovery - you're using store & forward.
As a quick proof of concept, I'd set up an email server, and build an application that sends emails to addresses registered on that server, encrypted using PGP - the tooling and libraries are available, so you should be able to get that up and running in days, rather than weeks. In my experience, building a throw-away PoC for this kind of question is a great way of sifting out the nugget of my idea.
The second issue is that the nature of a social network is that it's a network. Your design may require you to store more than the data of the two direct contacts - you may also have to store their friends, or at least the public interactions those friends have had.
This may not be part of your plan, but if it is, you need to think it through early on - you may end up having to transmit the entire social graph to each participant for local storage, which creates a scalability problem....
The paper about Safebook might be interesting for you.
Also you could take a look at other distributed OSN and see what they are doing.
None of the federated networks mentioned on http://en.wikipedia.org/wiki/Distributed_social_network is actually distributed. What Stefan intends to do is indeed new and was only explored by some proprietary folks.
I've been thinking about the same concept for the last two years. I've finally decided to give it a try using Python.
I've spent the better part of last night and this morning writing a sockets communication script & server. I also plan to remove the central server from the equation as it's just plain cumbersome and there's no point to it when all the members could keep copies of their friend's keys.
Each profile could be accessed via a hashed string of someone's public key. My social network relies on nodes and pods. Pods are computers which have their ports open to the network. They help with relaying traffic as most firewalls block incoming socket requests. Nodes store information and share it with other nodes. Each node will get a directory of active pods which may be used to relay their traffic.
The PeerSoN project looks like something you might be interested in: http://www.peerson.net/index.shtml
They have done a lot of research and the papers are available on their site.
Some thoughts about it:
protocols to use: you could think exactly on P2P programs and their design
security concerns: privacy. Take a great care to not open doors: a whole system can get compromised 'cause you have opened some door.
services: you could integrate with the regular social networks through their APIs
People will have to install a program in their computers and remeber to open it everytime, like any P2P client. Leaving everything on a web-server has a smaller footprint / necessity of user action.
Somehow you'll need a centralized server to manage the searches. You can't just broadcast the internet to find friends. Or you'll have to rely uppon email requests to add somenone, and to do that you'll need to know the email in advance.
The fewer friends /contacts use your program, the fewer ones will want to use it, since it won't have contact information available.
I see that your server will be a store and forward, so the update problem is solved.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
This question talks about different payment processors and what they cost, but I'm looking for the answer to what do I need to do if I want to accept credit card payments?
Assume I need to store credit card numbers for customers, so that the obvious solution of relying on the credit card processor to do the heavy lifting is not available.
PCI Data Security, which is apparently the standard for storing credit card info, has a bunch of general requirements, but how does one implement them?
And what about the vendors, like Visa, who have their own best practices?
Do I need to have keyfob access to the machine? What about physically protecting it from hackers in the building? Or even what if someone got their hands on the backup files with the sql server data files on it?
What about backups? Are there other physical copies of that data around?
Tip: If you get a merchant account, you should negotiate that they charge you "interchange-plus" instead of tiered pricing. With tiered pricing, they will charge you different rates based on what type of Visa/MC is used -- ie. they charge you more for cards with big rewards attached to them. Interchange plus billing means you only pay the processor what Visa/MC charges them, plus a flat fee. (Amex and Discover charge their own rates directly to merchants, so this doesn't apply to those cards. You'll find Amex rates to be in the 3% range and Discover could be as low as 1%. Visa/MC is in the 2% range). This service is supposed to do the negotiation for you (I haven't used it, this is not an ad, and I'm not affiliated with the website, but this service is greatly needed.)
This blog post gives a complete rundown of handling credit cards (specifically for the UK).
Perhaps I phrased the question wrong, but I'm looking for tips like these:
Use SecurID or eToken to add an additional password layer to the physical box.
Make sure the box is in a room with a physical lock or keycode combination.
I went through this process not to long ago with a company I worked for and I plan on going through it again soon with my own business. If you have some network technical knowledge, it really isn't that bad. Otherwise you will be better off using Paypal or another type of service.
The process starts by getting a merchant account setup and tied to your bank account. You may want to check with your bank, because a lot of major banks provide merchant services. You may be able to get deals, because you are already a customer of theirs, but if not, then you can shop around. If you plan on accepting Discover or American Express, those will be separate, because they provide the merchant services for their cards, no getting around this. There are other special cases also. This is an application process, be prepared.
Next you will want to purchase an SSL certificate that you can use for securing your communications for when the credit card info is transmitted over public networks. There are plenty of vendors, but my rule of thumb is to pick one that is a brand name in a way. The better they are known, the better your customer has probably heard of them.
Next you will want to find a payment gateway to use with your site. Although this can be optional depending on how big you are, but majority of the time it won't be. You will need one. The payment gateway vendors provide a way to talk to the Internet Gateway API that you will communicate with. Most vendors provide HTTP or TCP/IP communication with their API. They will process the credit card information on your behalf. Two vendors are Authorize.Net and PayFlow Pro. The link I provide below has some more information on other vendors.
Now what? For starters there are guidelines on what your application has to adhere to for transmitting the transactions. During the process of getting everything setup, someone will look at your site or application and make sure you are adhering to the guidelines, like using SSL and that you have terms of use and policy documentation on what the information the user is giving you is used for. Don't steal this from another site. Come up with your own, hire a lawyer if you need to. Most of these things fall under the PCI Data Security link Michael provided in his question.
If you plan on storing the credit card numbers, then you better be prepared to put some security measures in place internally to protect the info. Make sure the server the information is stored on is only accessible to members who need to have access. Like any good security, you do things in layers. The more layers you put in place the better. If you want you can use key fob type security, like SecureID or eToken to protect the room the server is in. If you can't afford the key fob route, then use the two key method. Allow a person who has access to the room to sign out a key, which goes along with a key they already carry. They will need both keys to access the room. Next you protect the communication to the server with policies. My policy is that the only thing communicating to it over the network is the application and that information is encrypted. The server should not be accessible in any other form. For backups, I use truecrypt to encrypt the volumes the backups will be saved to. Anytime the data is removed or stored somewhere else, then again you use truecrypt to encrypt the volume the data is on. Basically where ever the data is, it needs to be encrypted. Make sure all processes for getting at the data carries auditing trails. use logs for access to the server room, use cameras if you can, etc... Another measure is to encrypt the credit card information in the database. This makes sure that the data can only be viewed in your application where you can enforce who sees the information.
I use pfsense for my firewall. I run it off a compact flash card and have two servers setup. One is for fail over for redundancy.
I found this blog post by Rick Strahl which helped tremendously to understand doing e-commerce and what it takes to accept credit cards through a web application.
Well, this turned out to be a long answer. I hope these tips help.
Ask yourself the following question: why do you want to store credit card numbers in the first place? Chances are that you don't. In fact, if you do store them and manage to have one stolen, you could be looking at some serious liability.
I've written an app that does store credit card numbers (since the transactions were processed offline). Here's a good way to do it:
Get an SSL certificate!
Create a form to get CC# from the user.
Encrypt part (not all!) of the CC# and store it in your database. (I'd suggest the middle 8 digits.) Use a strong encryption method and a secret key.
Mail the remainder of the CC# to whoever processes your transactions (probably yourself) with the ID of the person to process.
When you log in later, you will type in the ID and the mailed-out portion of the CC#. Your system can decrypt the other portion and recombine to get the full number so you can process the transaction.
Finally, delete the online record. My paranoid solution was to overwrite the record with random data before deletion, to remove the possibility of an undelete.
This sounds like a lot of work, but by never recording a complete CC# anywhere, you make it extremely hard for a hacker to find anything of value on your webserver. Trust me, it's worth the peace of mind.
The PCI 1.2 document just came out. It gives a process for how to implement PCI compliance along with the requirements. You can find the full doc here:
https://www.pcisecuritystandards.org/security_standards/pci_dss.shtml
Long story short, create a separate network segment for whichever servers will be dedicated to storing CC info (usually DB server(s)). Isolate the data as much as possible, and ensure only the minimum access necessary to access the data is present. Encrypt it when you store it. Never store PAN's. Purge old data and rotate your encryption keys.
Example Don'ts :
Don't let the same account that can lookup general info in the database look up CC info.
Don't keep your CC database on the same physical server as your web server.
Don't allow external (Internet) traffic into your CC database network segment.
Example Dos:
Use a separate Database account to query CC info.
Disallow all but required traffic to CC database server via firewall/access-lists
Restrict access to CC server to a limited set of authorized users.
I'd like to add a non-technical comment that you may wish to think about
Several of my clients run e-commerce sites, including a couple who have moderately large stores. Both of those, whilst they certainly could implement a payment gateway choose not too, they take the cc number, store it temporarily encrypted online and process it manually.
They do this because of the high incidence of fraud and manual processing allows them to take additional checks before filling an order. I'm told that they reject a little over 20% of all their transactions - processing manually certainly takes extra time and in one case they have an employee who does nothing but process transactions, but the cost of paying his salary is apparently less than their exposure if they just passed cc numbers though an online gateway.
Both of these clients are delivering physical goods with resale value, so are particularly exposed and for items like software where a fraudulent sale wouldn't result in any actual loss your mileage would vary, but it's worth considering above the technical aspects of an online gateway if implementing such is really what you want.
EDIT: And since creating this answer I'd like to add a cautionary tale and say that the time is past when this was a good idea.
Why? Because I know of another contact who was taking a similar approach. The card details were stored encrypted, the website was accessed by SSL, and the numbers were deleted immediately after processing. Secure you think?
No - one machine on their network got infected by a key logging Trojan. As a result they were identified as being the source for several score credit card forgeries - and were consequently hit by a large fine.
As a result of this I now never advise anyone to handle credit cards themselves. Payment gateways have since become much more competitive and cost effective, and fraud measures have improved. The risk is now no longer worth it.
I could delete this answer, but I think best to leave up edited as a cautionary tale.
Keep in mind that using SSL to send a card number from a browser to a server is like covering your credit card number with your thumb when you hand your card to a cashier in a restaurant: your thumb (SSL) prevents other customers in the restaurant (the Net) from seeing the card, but once the card is in the hands of the cashier (a web server) the card is no longer protected by the SSL exchange, and the cashier could be doing anything with that card. Access to a saved card number can only be stopped by the security on the web server. Ie, most card thefts on the net aren't done during transmission, they're done by breaking through poor server security and stealing databases.
Why bother with PCI compliance?? At best you'll shave a fraction of a percent off your processing fees. This is one of those cases where you gotta be sure this is what you want to be doing with your time both upfront in development and over time in keeping up with the latest requirements.
In our case, it made the most sense to use a subscription-savy gateway and pair that with a merchant account. The subscription-savy gateway allows you to skip all the PCI compliance and do nothing more than process the transaction proper.
We use TrustCommerce as our gateway and are happy with their service/pricing. They have code for a bunch of languages that makes integration pretty easy.
Be sure to get a handle on the extra work and budget required for PCI. PCI may require huge external audit fees and internal effort/support. Also be aware of the fines/penalties that can be unilaterally levied on you, often hugely disproportionate to the scale of the 'ofense'.
There's a lot to the whole process. The single easiest way to do it is to use services similar to paypal, so that you never actually handle any credit card data. Apart from that, there's a quite a bit of stuff to go through to get approved to offer credit card services on your website. You should probably talk with your bank, and the people who issue your merchant ID to help you in setting up the process.
As others have mentioned the easiest way into this area is with the use of Paypal, Google checkout or Nochex. However if you intend to to a significant amount of business you may wish to look up "upgrading" to higher level site integrations services such as WorldPay, NetBanx (UK) or Neteller (US). All of these services are reasonably easy to set up. And I know that Netbanx offers convenient integration into some of the off the shelf shopping cart solutions such as Intershop (because I wrote some of them). Beyond that you are looking at direct integration with the banking systems (and their APAX systems) but thats hard and at that point you also need to prove to the Credit card companies that you are handling the credit card numbers securely (probably not worth considering if you are not taking $100k's worth per month).
Working from 1st to last the cost/benefits are that the early options are much easier (quicker/cheaper) to set up put you pay quite high handling charges for each transaction. the later ones are much more costly to set up but you pay less in the long run.
The other advantage of the most of the non dedicated solutions is that you don't need to keep encrypted credit card numbers secure. Thats someone else's problem :-)