all. I've started developing small extensions using Chrome's various API's, and although things are working great, I'm still curious about a few things.
Two questions, if you all wouldn't mind helping me out:
1. Could someone tell me what the limits are to chrome.storage.local's persistence? I've tried various tests on my own, such as storing a few things with chrome.storage.local.set(), clearing all of my browser history, cookies, etc. and then seeing if everything is still there. Often it will still be there, but sometimes I'll check back later and it will be gone. Overall, I have been unable to definitively label what I'm doing that is occasionally clearing the .local data.
2. I've been working mostly with chrome.storage.sync so far, all while not being signed in using Chrome's "Sign in to Chrome" feature. I read on the API page that in the case that .sync is used while Chrome is offline (as well as not logged in, I'm assuming, which is my case), data is stored locally and then sync'ed later on. My main concern is does this mean the data I'm storing using chrome.storage.sync could potentially get erased as the data I'm storing using chrome.storage.local has in the past? One of the main reasons I've been using .sync anyway is because I've never had an experience of data getting erased with it, while I have with .local (as I've described in #1).
Thanks so much! Help me out, please!
Edit: I'm pretty sure the .local clear isn't happening because of me mistakenly removing the extension and then adding it back in. I know that that will clear the .local data (but preserve the .sync).
I'm the author of that API.
chrome.storage.local shouldn't be disappearing except on uninstallation (which sounds like is your case) or, very rarely, on database corruption (and we've seen this particularly happening on System Restore).
chrome.storage.sync works the same way, except that the merge algorithm it uses may cause data loss if 2 machines make conflicting changes. In your case, this might happen if you sign into the machine which is using chrome.storage.sync. More commonly it will be because one machine is offline while making the change, or perhaps the user managed to simultaneously change data on 2 machines (which is why it's recommended to only change data on user action -- we should document that).
For what it's worth -- and we should document this too -- the merge algorithm is last-change-wins and sync-is-source-of-truth -- but any local key/value pairs added won't be deleted. If you have:
{a:1, b:2} on computer A (signed in and syncing),
{b:3, c:4} on computer B (not signed in),
and computer B signs in, after doing a full sync the state of storage on both A and B will end up at {a:1, b:2, c:4} because A's data was already part of sync, this the source of truth, but 'c' didn't exist yet so was added.
In this scenario A will have gotten an onChange event adding 'c', and B will have gotten an onChange event adding 'a' and updating 'b' from 3 to 2.
I spent some time looking at this today for work. My results are at https://github.com/mozilla/application-services/issues/2900#issuecomment-612251230. The behavior of chrome.storage.sync seems to have changed since the accepted answer. In particular, there doesn't appear to be any merging behavior any more. Instead it's all-or-nothing, with whatever object is on the sync server "winning". Deletes are transmitted to the server but not from the server to the other clients.
Related
I have some code I would like to find a vulnerability in, I coded it, and I would be grateful if you could help me out with finding anything in it.
My one and only concern are that getfenv may be able to be spoofed, in some way.
coroutine.wrap(function()
while wait() do
for i, v in func_table.pairs(func_table) do
if func_table.getfenv()[i] ~= v then
return ban_func(10, 23)
end
end
end
end)()
To be clear, the ban_func is inside the func_table, this will automatically detect its change in data and will ban accordingly. The only way I think they, being the exploiter/cheater, would be able to change anything is by spoofing getfenv.
If you could explain to me how it would be possible to spoof such a function and/or how to patch a spoof on the function, all without changing any of its own data, I would be very happy!
I assume that this code is running on the exploiter/cheaters machine. Fundamentally, there is no way to guarantee security of client code. Your checks can be removed, and your checks checks can be removed. Even the Lua binary itself can be changed internally and the getfenv can be changed to do anything in addition to what it does. Implementing a strong border between server and client logic is the only true way to secure applications.
One attack possible in this case is if the client runs Lua code in the same lua_State before your func_table is setup. In this case, they could sandbox you like my lua sandbox implementation found here.
Another attack is taking advantage of metamethods to make func_table.getfenv()[i] ~= v return true. This could be detected by using rawequal, checking the type of func_table.getfenv()[i], or using the original functions as keys in a table of true values and checking if the table at key func_table.getfenv()[i] is true.
Yet another attack would be to edit both the global state and your table. It is common when changing the address of a function, to change ALL references to that address in ram, which would include the internal reference inside your table.
Since you are using wait() I assume that you are running this code in Roblox. In that case, as I've emphasized on their developer forums, the experimental mode (previously filtering enabled) setting is the only way to secure your game from exploiters. This prevents clients from editing the game most of the time. They still have full control of their character position, a couple other instance types (like Hats I believe), can ask the server to make changes through RemoteEvent and RemoteFunction instances, and wall hack (set wall transparency) (to counter this, only send client parts that It can see).
As of late, I have encountered a problem with my view index being rebuilt all the time and users are having massive issues with this particular view.
I figured it was due to #Date in my selection formula aswell as one of my column formulas. This way the selection formula would be different every second that passes.
So I figured, since I dont need hours/minutes/seconds in my formulas, I would use #Today. This worked out well for 2-3 days and after that the same problem occured again.
So since the problem is back again, I'm not quite sure if that even causes the problem. When this particular view is open, I have issues in every tab that's open in notes, not only this specific database.
Is this a common/known issue? What can I do to avoid this problem?
Yes, it's a common issue that has been well known since the very early days of Notes more than 20 years ago.
#Date is not a problem on its own. #Now and #Today are both problems.
Using #TextToTime("Today") was a popular workaround that was discovered early on. This hid the problem from the indexer so the server failed to realize that the view was out of date. It doesn't solve the underlying problem, though, which is that the view is trying to do something that views simply aren't designed to do. Views are intended to be static, requiring update only when documents change. Introducing time into a selection or column formula makes them dynamic, which kills that presumption and is a major source of performance problems. Using this workaround requires that the view be fully rebuilt every night. You can do that by setting the view index options to "Manual", and setting up a program document to run an updall command with the -T option for the specific database and view once per night. Note that if your users are spread out across timezones, you'll have to pick one specific time as your standard, and if you have servers spread out across timezones you're going to have a lot of fun figuring out how to make them all show the same documents in the view at all times - but that's common to pretty much all approaches to the problem.
See this IBM Technote for a description of several other options that people have used over the years, with their pros and cons. Also see this article by Andre Guirard, which covers date/time issues in great detail.
I would add that the agent-and-folder solution that they describe in the Technote was generally my preferred approach, but it does have an additional disadvantage that they don't mention: it can eventually lead to an obscure situation where the server throws an error "Folder is larger than supported". This error actually has nothing to do with the size of the folder in documents; it refers to fragmentation of internal structures that occurs as large numbers of documents are moved in and out of the folder over time. It could only be fixed by deleting and re-creating the folder, which you can do in your agent code. I believe this problem may be fixed in more recent versions of Domino, but it caused me a lot of grief back in the Notes 6 and 7 timeframes.
Yet another weird story from Domino Designer 9.0.1:
The application in question is set to support German and English; German is set to be both the source and the default langauge.
Over the course of the past few weeks we observed that there are some CustomControls and Xpages whose properties files are multiplying; within something like 12 hours we often see hundreds of multiplied files (currently we have 120 multiplications; earlier this week we had a case with > 1000 multiplied propertiey files!) In package explorer they turn up like this:
As you can see there is something like a docUnid added to the property's file name. Apart from a different time stamp they all are identical internally. In same cases both language versions are multiplied, in this particular case here only the German (= source) version shows that phenomenon.
Another strange fact: this particular custom control hasn't changed for quite a while, and it only contains a single control with a static text attribute, alongside a
Anyone having an idea what could be causing this, and what possible solutions I could try?
Tech facts and some more observations:
Domino Designer 9.0.1 FP6, ExtLib 17; we are working in a team where each one of us is coding in their own local replica, then replicating into the "hub" replica. I can't prove it but I assume that there is a connection between one of us replicating updates and the creation of new prüperty duplicates
EDIT: some more observations: I think I was able to pin it down to the replication between two specific machines; I just ran a sequence of 5 or 6 manually driven replications between both instances, every time without making any changes to the design code on either side. nevertheless every replication reported exactly 1 update and 1 addition, and each time a new property file was added.
So meanwhile I deleted the custom control in question and rebuilt it from scratch under a slightly different name (just to be on the safe side). For now it seems that the application is "behaving" now but I'm sort of sure that this will return sooner or later.
Speak after me: source control and replication do not match.
More details:
The property files get stored as attachments in a design note. That's usually the note with the form. Unless you switch on multi lingual, then each property gets its own note. When different people work on the database these note elements get recreated on build getting the next UNID kind of.
So the right flow for what you try to do: pick your best version of the nsf. Nuke the other replicas. Bind it to version control. Let your peer developer create an nsf from that repository. Sync of design shall only happen via that repository.
While your on it: add Bavarian as language, so your Munich customers can use the app too
Using the Instagram API, I subscribed to a tag with the Real time feature. I sync media that match my project's criteria, then save those to DB. When users visit my website, I display these images from my DB (and not from instagram API).
From time to time, I see broken links showing up in the images. I identified that the source of the problem is that those images have now been deleted.
What's a good way to handle this?
Probably not attempting to duplicate the Instagram DB (or part thereof) would be the best option. Depending on the usage of your project and what sort of tags you're subscribing to, that could get pretty large pretty quickly.
Short of that, doing a quick HTTPRequest to the image URL (and checking the response code) before deciding whether to display it would do the job.
#Steve Crawford is on the right track.
The problem with your solution is that you are duplicating volatile data that you:
a) can't control
b) don't receive notifications on.
I would think the better method would be track the meta data of the images you are interested (like the author, url,date,etc) and then display them if they are still available.
If you are going to cache data you also need to a way to invalidate your cache. So another option would be to duplicate the data as you already are, but also have a background job to ensure that the data is still valid and remove the ones that aren't.
How can I prevent that forms can be scanned with a sort of massive vulnerability scanners like XSSME, SQLinjectMe (those two are free Firefox add-ons), Accunetix Web Scanner and others?
These "web vulnerability scanners" work catching a copy of a form with all its fields and sending thousands of tests in minutes, introducing all kind of malicious strings in the fields.
Even if you sanitize very well your input, there is a speed response delay in the server, and sometimes if the form sends e-mail, you vill receive thousands of emails in the receiver mailbox. I know that one way to reduce this problem is the use of a CAPTCHA component, but sometimes this kind of component is too much for some types of forms and delays the user response (as an example a login/password form).
Any suggestion?
Thanks in advance and sorry for my English!
Hmm, if this is a major problem you could add a server-side submission-rate limiter. When someone submits a form, store some information in a database about their IP address and what time they submitted the form. Then whenever someone submits the form, check the database to see if it's been "long enough" since the last time that IP address submitted the form. Even a fairly short wait like 10 seconds would seriously slow down this sort of automated probing. This database could be automatically cleared out every day/hour/whatever, you don't need to keep the data around for long.
Of course someone with access to a botnet could avoid this limiter, but if your site is under attack by a large botnet you probably have larger problems than this.
On top the rate-limiting solutions that others have offered, you may also want to implement some logging or auditing on sensitive pages and forms to make sure that your rate limiting actually works. It could be something simple like just logging request counts per IP. Then you can send yourself an hourly or daily digest to keep an eye on things without having to repeatedly check your site.
Theres only so much you can do... "Where theres a will theres a way", anything that you want the user to do can be automated and abused. You need to find a median when developing, and toss in a few things that may make it harder for abuse.
One thing you can do is sign the form with a hash, for example if the form is there for sending a message to another user you can do this:
hash = md5(userid + action + salt)
then when you actually process the response you would do
if (hash == md5(userid + action + salt))
This prevents the abuser from injecting 1000's of user id's and easily spamming your system. Its just another loop for the attacker to jump through.
Id love to hear other peoples techniques. CAPTCHA's should be used on entry points like registration. And the method above should be used on actions to specific things (messaging, voting, ...).
also you could create a flagging system, and anything the user does X times in X amount of time that may look fishy would flag the user, and make them do a CAPTCHA (once they enter it they are no longer flagged).
This question is not exactly like the other questions about captchas but I think reading them if you haven't already would be worthwhile. "Honey Pot Captcha" sounds like it might work for you.
Practical non-image based CAPTCHA approaches?
What can be done to prevent spam in forum-like apps?
Reviewing all the answers I had made one solution customized for my case with a little bit of each one:
I checked again the behavior of the known vulnerability scanners. They load the page one time and with the information gathered they start to submit it changing the content of the fields with malicious scripts in order to verify certain types of vulnerabilities.
But: What if we sign the form? How? Creating a hidden field with a random content stored in the Session object. If the value is submitted more than n times we just create it again. We only have to check if it matches, and if it don't just take the actions we want.
But we can do it even better: Why instead to change the value of the field, we change the name of the field randomly? Yes changing the name of the field randomly and storing it in the session object is maybe a more tricky solution, because the form is always different, and the vulnerability scanners just load it once. If we don’t get input for a field with the stored name, simply we don't process the form.
I think this can save a lot of CPU cycles. I was doing some test with the vulnerability scanners mentioned in the question and it works perfectly!
Well, thanks a lot to all of you, as a said before this solution was made with a little bit of each answer.