Is there a way to train an llm model to store a specific context? For example, I had a long story I want to ask questions about, but I don't want to put the whole story in every prompt. How can I make the llm model "remember the story"?
Taking into account that GPT-3 models have no parameter that enables memorization of past conversations, it seems the only way at the moment to "memorize" past conversations is to include past conversations in the prompt.
If we take a look at the following example:
You are a friendly support person. The customer will ask you questions, and you will provide polite responses
Q: My phone won't start. What do I do? <-- This is a past question
A: Try plugging your phone into the charger for an hour and then turn it on. The most common cause for a phone not starting is that the battery is dead.
Q: I've tried that. What else can I try? <-- This is a past question
A: Hold the button in for 15 seconds. It may need a reset.
Q: I did that. It worked, but the screen is blank. <-- This is a current question
A:
Rule to follow:
Include prompt-completion pairs in the prompt with the oldest conversations at the top.
Problem you will face:
You will hit a token limit at some point (if you chat long enough). Each GPT-3 model has a maximum number of tokens you can pass to it. In the case of text-davinci-003, it is 4096 tokens. When you hit this limit, the OpenAI API will start to throw errors. When this happens, you need to reduce the number of past prompt-completion pairs (e.g., include only the most recent 4 past prompt-completion pairs).
Pros:
By including past prompt-completion pairs in the prompt, we are able to give
GPT-3 models the context of the conversation.
Cons:
What if a user asks a question that relates to a conversation that occurred more than 4 prompt-completion pairs ago?
Including past prompt-completion pairs in the prompt will cost (a lot of) money!
Related
I'm creating a chatbot in Dialogflow in which the user is expected to enter a frequency of time, followed by specifying the time. i.e :
Bot: how many entries will you make on that day? (Or what so the frequency of your entries)?
User: Twice daily or two times a day.
Bot: Please enter those times.
User: 9 am and 7 pm
Now the problem is even if I enter more than two times it will still get accepted as the time by Dialogflow.
I need to implement a check here that will take only times if the user enters twice daily and accept three times if the frequency is thrice daily.
Is it possible to do this by manipulating entities and intents? I want to avoid doing this in the webhook.
Also the webhook I'll be implementing is in python. So can't use Node.js inline editor.
No, this can't be done in the Intents only. Remember that an Intent represents what the user has said, and not how you are using.
As you surmise, the best place to check these values are in your webhook fulfillment. Since you already have a webhook, it isn't clear why you are avoiding this.
In terms of design, you may wish to skip asking for the frequency and just ask the user to tell you when they'll be making the entries. You can then confirm that was all that they wanted, accept more if they needed more, etc.
At a high level, I would like to set up an action that has some required inputs and some optional ones. After the user begins, he/she will be prompted for any required inputs that are missing. If/when the required inputs are collected, i would like to ask if the user wants to specify more optional inputs.
The specific use case is a voice-based real estate search. I have some required inputs set up like zip code, price, and number of bedrooms. Then I would like bixby to ask "would you like to refine your search even further?" and if the user says yes, we can ask about number of bathrooms, parking arrangements, and other more niche parameters. I do not want to make all of these required and have to prompt everyone if they want to filter by "pools" or some parameter not widely used. And being voice-based, I do not want to just have it as a checkbox on the screen because someone on the speaker won't be able to use that parameter then.
I have thought of 2 potential solutions but I do not know if they will work (at least without relying on the controls on screen for a voice-based capsule):
1) Make the search into a transaction and then instead of a normal confirmation, try to shoe-horn the confirmation to ask if the user wants to add more refinements. Maybe something like the bank transfer one but a negative confirmation would cause bixby to ask for information that she didnt ask for before: https://bixbydevelopers.com/dev/docs/sample-capsules/walkthroughs/simple-transactional#sample-capsule-walkthrough
2) Make 2 more required inputs, one boolean called "WantsOptionalParameters" and the other called "OptionalParameters" that will be a structure containing all of the optional parameters. Then it would prompt for WantsOptionalParameters like a normal required input, and if that is "True", I can have a sub-action that will ask for each one of the parameters to construct an OptionalParameters object. then we could feed that output into the search. And alternatively, if WantsOptionalParameters is false, we can automatically construct OptionalParameters with all negative responses and feed that into the original action.
Both of these solutions will take a bunch more research and testing and i don't even know if they will work, so i was hoping to call on the wisdom of you guys!
Here is my take on it for what its worth. Every domain has key inputs that are typically used to start the conversation and optional inputs that can refine the conversation.
Some general ways to start conversation for the real estate example (totally driven by my own experience, perhaps there are more)
Hows the real estate market in 90210?
Show me homes under $250K in Los Angeles?
Show me homes with 4 bedrooms (near me?)
You can club such inputs into an input-group called RequiredInputs that requires OneOrMoreOf these parameters to get the capsule started.
You can also collect the optional/niche inputs in another input-group called OptionalInputs that requires ZeroOrMoreOf and feed them into your capsule logic
Its also possible that all inputs are equally important and are all Optional! It is totally dependent on the domain and how the capsule developer might handle such inputs.
But in a general sense, once the set of inputs is in and the initial results are shown to the user, the capsule developer then has a great amount of control to
Shape the future conversation AND
Highlight capsule capabilities.
So, rather than presenting the user with a set of options, you can control the conversation and offer options that provide most value to the user (and to the capsule developer!)
e.g your capsule is capable of deep analyzing and refining results in a way that no other capsule on the market can do. So, you want to highlight this capability as the first choice via followup
Or You may have a conversation path based on prior experiences and your knowledge of the domain. So, you could say, I can refine the results further by X, Y, Z options.
This scenario is more likely to be useful and less likely to overwhelm (with options) the end user.
Hope this helps!
I plan on creating an ticket "pass" platform. Basically, imagine you come to a specific city, you buy a "pass" for several days (for which you get things like free entrance to museums and other attractions).
Now, the main question that bothered me for several days is: How will museum staff VALIDATE if the pass is valid? I see platforms like EventBrite etc. using barcodes/QR codes, but that is not quite a viable solution because we'll need to get a good camera phone for every museum to scan the code and that's over-budget. So I was thinking of something like a simple 6-letter code, for eg: GHY-AGF. There are 26^6 = 308 million combinations, which is a tough nut to crack.
I've asked a question on the StackExchange security site about this, and the main concern was the brute forcing. However, I imagine someone doing this kind of attack if: they had access of doing pass lookup. The only people that will be able to do this are:
1) The museum staff (for which there will be a secure user/pass app, and rate limits of no more than 1000 look-ups per day)
b) Actual customers to check the validity of their pass, and this will be protected with Google ReCaptcha v3, which doesn't sacrifice user experience like with v1. Also rate limits and IP bans will be applied
Is a brute force STILL a viable attack if I implement these 2 measures in place? Also, is there something else I'm missing in terms of security, when using this approach?
By the way, Using a max. 6-character-long string as a unique "pass" has many advantages portable-wise, for eg. you could print "blank" passes, where the user will be give instructions on how to obtain it. After they pay, they'll be given a code like: GAS-GFS, which they can easily write with a pen on the pass. This is not possible with a QR/barcode. Also, the staff can check the validity in less than 10 seconds, by typing it in a web-app, or sending an SMS to check if it's valid. If you're aware of any other portable system like this, that may be more secure, let me know.
Brute forcing is a function of sparseness. How many codes at any given time are valid out of how large a space? For example, if out of your 308M possibilities, 10M are valid (for a given museum), then I only need ~30 guesses to hit a collision. If only 1000 are valid, then I need more like 300k guesses. If those values are valid indefinitely, I should expect to hit one in less than a year at 1000/day. It depends on how much they're worth to figure out if that's something anyone would do.
This whole question is around orders of magnitude. You want as many as you can get away with. 7 characters would be better than 6 (exactly 26x better). 8 would be better than that. It depends on how devoted your attackers are and how big the window is.
But that's how you think about the problem to choose your space.
What's much more important is making sure that codes can't be reused, and are limited to a single venue. In all problems like this, reconciliation (i.e. keeping track of what's been issued and what's been used) is more important than brute-force protection. Posting a number online and having everyone use it is dramatically simpler than making millions of guesses.
I'm developing a feature for a client in which users voluntarily take an important test online. The test is difficult and the users will be highly motivated to do well (think SATs or GRE, etc)... so there's also a high incentive to cheat. Apparently there are 3rd party services in which a human virtually monitors the test taker via a webcam, but they're really expensive and we don't quite have the budget. We still need to make it as hard as possible for a user to game the system. Some of the things we suspect they might try are:
Getting someone else to take the test for them (a pinch hitter).
Taking the test multiple times with different profiles to practice
and gain an unfair advantage.
Taking the test alongside friends or while in contact with a friends
to tell them the answers.
The question order will change, as well as the order of the answers. The test will be timed, and an "open book" format, so we're not really worried about the user looking things up online, but we can't have them sharing their screen and having others assist them. So the main concern at this point is ensuring that the user is, in fact, who they say they are (and not someone else).
Here are a few of the security measures we're considering:
Requiring the user's device to have a webcam, which we'll activate and either record/photograph the user during the test (with the user's consent of course).
Asking users to verify an arbitrary bank deposit amount (presumably via PayPal). There's nothing to stop them from opening up multiple bank accounts, but at least it's a big hassle.
Really scary terms of use that threaten legal action if the user is caught cheating.
QUESTION: Are there any other measure we can/should take to make sure our test is secure and the results are reliable?
CLARIFICATION: We realize that with enough resources and determination, any security system can eventually be beaten. The goal of this question is not to find a magically unbeatable solution, but to find ways to raise the stakes enough so that it won't be worth it for most users to cheat. In this spirit, I'd much prefer answers that focus on what can be done as opposed to what can't.
As you know there are many ways of cheating. Your goal is limit the possibility of cheating as much as possible. Cheating in online courses has been a hot topic.
A pinch hitter:
This type of attack can be conducted a number of ways. Even if you have a cam looking at the person, the video that the test taker is seeing could be mirrored on another screen. A pinch hitter could see the question and just read him the answers or otherwise feed answers the test taker in a covert channel.
Possible counters to this attack is to also enable the mic to see if they are talking to anyone. You can also record the screen while they take the test. This could prevent them from opening a chat window or viewing other unauthorized content. (Kind of like the Elance tracker)
user verification:
In order to register the person should attach a scanned copy of their photo-id. This way you are linking a photo of the person to a unique identifier, such as a drivers license number. Before the person starts taking the test, ask the user to look directly at the camera and make sure you get a good image of them that can be verified against their photo id.
A simple attack against this system is to use photoshop to modify the id. To make this attack more difficult you could verify their name against a credit/debit card transaction. The names should match on both cards.
An evercookie could be used to track machines to see if the same computer is being used. This could happen though legitimate reasons, but it could also be used to flag tests for further review. A variant on the evercookie is to drop a file with a random value or set a registry key with a random value to "mark" that machine.
I have a backend that generates gift codes, each with a certain number of uses. Give these to a blogger or whatever, and their readership can redeem the code for a promotional item.
I'm working on the best way to check a codes validity without having collisions/dupes, or anything like that. I need to 1) validate the code 2) collect shipping info
My first draft was
A) Check code via a form, if good, proceed to address input. When input is received, save code and address/name etc.
This fails because if there are 74 uses on a 75 use code, 25 people could "validate" but not enter their address yet, and we'd end up with more than 75 valid redemptions.
My current solution looks more like:
B) Just have the code as the first field in the information gathering form, and when a valid code is typed in, ajaxify that and live check it against the DB. If the code is valid, it then shows the rest of the form, and that entry of the code is "claimed" for half an hour or something. If no DB entry w/in half an hour, it's then released.
This seems pretty complex, and I'm wondering if I'd need to do throttling against the ajax attempts to make sure people don't brute force a valid code.
Is this method secure, and/or are there any other blatantly obvious patterns I'm missing for this type of application?
Let everyone enter their gift-code and address, and then submit
In the backend, verify the address and the gift-code.
If the gift-code is valid and not exhausted, congratulate the user. Else apologise to them and suggest they buy it instead anyway.
Does it have to be more complicated than that?
Why don't you just have one form with all the information (redemption code and shipping info)?
Then, when the user submits, atomically (using transactions on your database) check if it's valid and commit the user's information.
If the code is no longer valid, just show a message like "Sorry, the redemption code you used has been depleted and is no longer valid."
Just wanted to add, if you're worried about bruteforcing attempts, you can require a captcha or javascript based hashcash value to be submitted along with the gift code. If you want to be as unobtrusive as possible, you can only require this for subsequent attempts after the first failed one.
One thing you might consider, is after the user enters a gift code, create an intermediate page that has more details about the offer, shows the number of claims remaining, and has some information about what will be required to complete the offer (address, creditcard, whatever). If the user chooses to claim the offer, have a 10-15 minute countdown (updated via javascript) on the data entry page for the address and other personal information, so the user knows that the offer might expire if they don't enter their information immediately.
Another thing to consider is implementing a "cancel" button that indicates the user can make the offer available for another user, without waiting for the countdown to expire.
Your current solution looks like the proper one, although I think you left out the method by which you associate the user association with the code. Still, providing the functionality of "reserving" a redemption of the code for a user is a good solution.
Option B seems reasonable. Just use a captcha rather than trying to throttle it. Captchas aren't perfect but it's less obnoxious than say misreading the code three times and then being denied the ability to try another for 24 hours. This will work particularly well if you're already planning on doing it AJAXy.
So -
User will fill in code field and captcha.
You'll confirm the captcha, then confrim the code.
Once successful, the user will fill in the other info and submit.
Using this method you could also probably only lock the code for something more like 5 minutes (ticket agency style) and show a timer on the form somewhere notifying the user.
Your A method (Check code via a form, if good, proceed to address input) looks very reasonable. Just combine it with B's "code is "claimed" for half an hour or something", and everything should work as you expect.
That is:
Customer enters code
Check code – if valid, and not already used MAX+ times, add an extra entry in code use table, with a timestamp that expires after x minutes.
Collect other info
On submit, permanently mark the code as used (remove entry expiration)
If customer never makes the order, the timestamped entry is removed (or ignored) after time x, and released for others to use.
We do a low end encryption (RC4) with a checksum added for this type of thing. Because RC4 generates a problematic character set, we also converted it to HEX. The combination is relatively secure and self checking. The decrypted value is just a number that we can verify in the database. This works with both our eMail reminders and gift certificates.