I am working on a anchor / solana program that provides liquidity to a number of pools, including saber.so and invariant.app. During the swap, I need to calculate the TVL, to provision a token at a fair exchange rate.
My question is: what is the best way to calculate TVL onchain?
The following are some approaches that I have in mind, each one with its shortcomings:
(1) Calculate off-chain, and provide this as an oracle:
We could calculate the TVL off-chain, and then provide this TVL as an oracle. The shortcomings are: chainlink (an oracle provider) on solana does not seem to support custom data-feeds, as is the case with ethereum.
Further more this solution increases the centralization of the app, it would be nice to have it on-chain. also there could be oracle-attacks which drain the reserves of the protocol.
(2) Have a giant list of liquidity-positions:
Another approach would be to keep track of all liquidity-positions that we as a protocol have provided liquidity in. Although this is possible, I believe that this would (very quickly), reach solana's account limit.
In this case, we would have a huge "state-"account, which tracks the following variables per pool:
token1_mint: Pubkey
token2_mint: Pubkey
token1_amount: u64
token2_amount: u64
token1_to_currency_pyth_feed_address: Pubkey
token2_to_currency_pyth_feed_address: Pubkey
provider: u8
Given that we have 4 * 32 + 2 * 64 bytes + 8 bytes = 264 bytes, we can have around 20 pools that we can deposit at any given point in time (because of a 4KB account limit on solana)
The second option seems like the way to go, as the first one if off-chain and prone to oracle-attacks. However, the second option still seems a bit hacky, as I would have to include this data-structure anytime I intend to calculate the total TVL.
Is there any other design ideas that come to your mind or that you have seen, that would be appropriate?
I don't know much about the overall design of your program to provide you with a good solution. I also don't know what invariant is, maybe that breaks what I'm about to describe below.
I assume that you have some instruction in your program which cpi calls into Saber etc and opens a liquidity position. Assuming that that instruction creates an account on the chain with the following information:
pool_address,
token_1_mint_address
token_2_mint_address
amount_token_1
amount_token_2
...
One simple solution is to loop through all those accounts, and since you have the amount and mint of each token, you can calculate the value using something like the pyth price oracle. I wouldn't do this on chain though since it can become pretty expensive fast! Perhaps is best to do it on the client side and write this information back to the chain.
The recent solana bootcamp videos actually have a tutorial on bringing off chain info back to the chain! https://www.youtube.com/watch?v=GwhRWde3Ckw&t=385s
Below is a demo of the runtime limitations of on chain programs, perhaps you could do the loop through the pool accounts, if you use some indexing and PDA to find the account address and assuming that you have a limited number of liquidity positions! However I wouldn't hardcode all the information into a single account, seems like an unsustainable approach that might cause a lot of issues down the line. Might give you superior performance however, not sure.
https://www.youtube.com/watch?v=5IrfSecDPeA&t=1191s
Anyways GL!
Related
IF, someone was able to retrieve the intentionally lost information from the SHA function by finding a vulnerability in the loose compression of data & able to get the input of every SHA256 output & verify a block that does not belong in the main BTC network, can't it be used to create a fake dust transaction and manually verify that transaction? - Ultimately creating an unlimited BTC in seconds?
Be aware that for a given SHA256 output there may be several inputs that result to the given SHA256 output.
AFAIK Bitcoin strongly relies on not being able to reverse SHA256 function. So without talking about faking blocks or making dust transactions, if you are able to reverse the SHA256, then you can instantly mine Bitcoin blocks making you reward all the remaining bitcoins not yet mined. However if such an extreme miner appeared, he would be spotted by the whole network and so the value of bitcoins would be lost as the network isn't working as it was designed.
I am trying to create a special military RADIO transmitter.
Basically, the flow is:
A solider will receive a message to transmit (about 10 times a day). Each message is of length 1024 bits exactly.
He will insert this message into the radio and validate it is inserted correctly.
The RADIO will repetitively transmit this message.
This is very important that the transmitter will not be hacked, because its very important in times of emergencies.
So, the assistance I ask from you is, how to preform stage 2 without risking getting infected.
If I will transfer the data using a DOK, it may be hacked.
If I will make the user type in 1024 bits, it will be safe but exhausting.
Any Ideas? (unlimited budget)
(It’s important to say that the data being transmitted is not a secret)
Thanks for any help you can supply!
Danny
Edit:
Basically, I want to create the most secure way to transfer a fixed number of bits (in this case 1024), from one (may be infected computer) to the other (air gaped computer).
without having any risk of a virus being transferred as well.
I don't mind if an hacker will change the data that is transferred from the infected computer, I just want that the length of the data will be exactly 1024, and avoiding virus to be inserted to the other computer.
Punch card (https://en.wikipedia.org/wiki/Punched_card) sounds like a good option, but an old one.
Any alternatives?
The transmitter is in the field, and is one dead soldier away from falling into enemy hands at any time. The enemy will take it apart, dissect it, learn how it works and use the protocol to send fraudulent messages that may contain exploit code to you, with or without the original equipment. You simply cannot prevent a trasmitter or otherwise mocked up "enemy" version of a transmitter from potentially transmitting bad stuff, because those are outside of your control. This is the old security adage "Never trust the client" taken to its most extreme level. Even punch cards could be tampered with.
Focus on what you can control: The receiving (or host) computer (which, contrary to your description, is not airgapped as it is receiving outside communication in your model) will need to validate the messages that come in from the client source; this validation will need to check for malicious messages and handle them safely (don't do anything functional with them, just log it, alert somebody and move on with life).
Your protocol should only be treating inbound messages like text or identifiers for message types. Under no circumstances should you be trying to interpret them as machine language instructions, and any SQL queries or strings that this message is appended to should be properly sanitized. This will prevent the host from executing any nasties that do come in.
I am trying to figure out a way to implement decent crypto on a micro-controller project. I have an ARMv4 based MCU that will control my garage door and receive commands over a WiFi module.
The MCU will run a TCP/IP server, that will listen for commands from Android clients that can connect from anywhere on the Internet, which is why I need to implement crypto.
I understand how to use AES with shared secret key to properly encrypt traffic, but I am finding it difficult to deal with Replay Attacks. All solutions I see so far have serious drawbacks.
There are two fundamental problems which prevent me from using well
established methods like session tokens, timestamps or nonces:
The MCU has no reliable source of entropy, so I can't generate
quality random numbers.
The attacker can reset the MCU by cutting power to the garage,
thus erasing any stored state at will, and resetting time counter to
zero (or just wait 49 days until it loops).
With these restrictions in mind, I can see only one approach that seems
ok to me (i.e. I don't see how to break it yet). Unfortunately, this
requires non-volatile storage, which means writing to external flash,
which introduces some serious complexity due to a variety of technical details.
I would love to get some feedback on the following solution. Even better, is there a solution I am missing that does not require non-volatile storage?
Please note that the primary purpose of this project is education. I realize that I could simplify this problem by setting up a secure relay inside my firewall, and let that handle Internet traffic, instead of exposing the MCU directly. But what would be the fun in that? ;)
= Proposed Solution =
A pair of shared AES keys will be used. One key to turn a Nonce into an IV for the CBC stage, and another for encrypting the messages themselves:
Shared message Key
Shared IV_Key
Here's a picture of what I am doing:
https://www.youtube.com/watch?v=JNsUrOVQKpE#t=10m11s
1) Android takes current time in milliseconds (Ti) (64-bit long) and
uses it as a nonce input into the CBC stage to encrypt the command:
a) IV(i) = AES_ECB(IV_Key, Ti)
b) Ci = AES_CBC(Key, IV(i), COMMAND)
2) Android utilizes /dev/random to generate the IV_Response that the
MCU will use to answer current request.
3) Android sends [<Ti, IV_Response, Ci>, <== HMAC(K)]
4) MCU receives and verifies integrity using HMAC, so attacker can't
modify plain text Ti.
5) MCU checks that Ti > T(i-1) stored in flash. This ensures that
recorded messages can't be replayed.
6) MCU calculates IV(i) = AES_ECB(IV_Key, Ti) and decrypts Ci.
7) MCU responds using AES_CBC(Key, IV_Response, RESPONSE)
8) MCU stores Ti in external flash memory.
Does this work? Is there a simpler approach?
EDIT: It was already shown in comments that this approach is vulnerable to a Delayed Playback Attack. If the attacker records and blocks messages from reaching the MCU, then the messages can be played back at any later time and still be considered valid, so this algorithm is no good.
As suggested by #owlstead, a challenge/response system is likely required. Unless I can find a way around that, I think I need to do the following:
Port or implement a decent PRGA. (Any recommendations?)
Pre-compute a lot of random seed values for the PRGA. A new seed will be used for every MCU restart. Assuming 128-bit seeds, 16K of storage buys be a 1000 unique seeds, so the values won't loop until the MCU successfully uses at least one PRGA output value and restarts a 1000 times. That doesn't seem too bad.
Use the output of PRGA to generate the challenges.
Does that sound about right?
Having an IV_KEY is unnecessary. IVs (and similar constructs, such as salts) do not need to be encrypted, and if you look at image you linked to in the youtube video you'll see their description of the payload includes the IV in plaintext. They are used so that the same plaintext does not encode to the same ciphertext under the same key every time, which presents information to an attacker. The protection against the IV being altered is the HMAC on the message, not the encryption. As such, you can remove that requirement. EDIT: This paragraph is incorrect, see discussion below. As noted though, your approach described above will work fine.
Your system does have a flaw though, namely the IV_Response. I assume, from that you include it in your system, that it serves some purpose. However, because it is not in any way encoded, it allows an attacker to respond affirmatively to a device's request without the MCU receiving it. Let's say that your device's were instructing an MCU that was running a small robot to throw a ball. Your commands might look like.
1) Move to loc (x,y).
2) Lower anchor to secure bot table.
3) Throw ball
Our attacker can allow messages 1 and 3 to pass as expected, and block 2 from getting to the MCU while still responding affirmatively, causing our bot to be damaged when it tosses the ball without being anchored. This does have an imperfect solution. Append the response (which should fit into a single block) to the command so that it is encrypted as well, and have the MCU respond with AES_ECB(Key, Response), which the device will verify. As such, the attacker will not be able to forge (feasibly) a valid response. Note that as you consider /dev/random untrustworthy this could provide an attacker with plaintext-ciphertext pairs, which can be used for linear cryptanalysis of the key provided an attacker has a large set of pairs to work with. As such, you'll need to change the key with some regularity.
Other than that, your approach looks good. Just remember it is crucial that you use the stored Ti to protect against the replay attack, and not the MCU's clock. Otherwise you run into synchronization problems.
I have created a small wireless network using a few PIC microcontrollers and nRF24L01 wireless RF modules. One of the PICs is PIC18F46K22 and it is used as the main controller which sends commands to all other PICs. All other (slave) microcontrollers are PIC16F1454, there are 5 of them so far. These slave controllers are attached to various devices (mostly lights). The main microcontroller is used to transmit commands to those devices, such as turn lights on or off. These devices also report the status of the attached devices back to the main controller witch then displays it on an LCD screen. This whole setup is working perfectly fine.
The problem is that anybody who has these cheap nRF24L01 modules could simply listen to the commands which are being sent by the main controller and then repeat them to control the devices.
Encrypting the commands wouldn’t be helpful as these are simple instructions and if encrypted they will always look the same, and one does not need to decrypt it to be able to retransmit the message.
So how would I implement a level of security in this system?
What you're trying to do is to prevent replay attacks. The general solution to this involves two things:
Include a timestamp and/or a running message number in all your messages. Reject messages that are too old or that arrive out of order.
Include a cryptographic message authentication code in each message. Reject any messages that don't have the correct MAC.
The MAC should be at least 64 bits long to prevent brute force forgery attempts. Yes, I know, that's a lot of bits for small messages, but try to resist the temptation to skimp on it. 48 bits might be tolerable, but 32 bits is definitely getting into risky territory, at least unless you implement some kind of rate limiting on incoming messages.
If you're also encrypting your messages, you may be able to save a few bytes by using an authenticated encryption mode such as SIV that combines the MAC with the initialization vector for the encryption. SIV is a pretty nice choice for encrypting small messages anyway, since it's designed to be quite "foolproof". If you don't need encryption, CMAC is a good choice for a MAC algorithm, and is also the MAC used internally by SIV.
Most MACs, including CMAC, are based on block ciphers such as AES, so you'll need to find an implementation of such a cipher for your microcontroller. A quick Google search turned up this question on electronics.SE about AES implementations for microcontrollers, as well as this blog post titled "Fast AES Implementation on PIC18F4550". There are also small block ciphers specifically designed for microcontrollers, but such ciphers tend to be less thoroughly analyzed than AES, and may harbor security weaknesses; if you can use AES, I would. Note that many MAC algorithms (as well as SIV mode) only use the block cipher in one direction; the decryption half of the block cipher is never used, and so need not be implemented.
The timestamp or message number should be long enough to keep it from wrapping around. However, there's a trick that can be used to avoid transmitting the entire number with each message: basically, you only send the lowest one or two bytes of the number, but you also include the higher bytes of the number in the MAC calculation (as associated data, if using SIV). When you receive a message, you reconstruct the higher bytes based on the transmitted value and the current time / last accepted message number and then verify the MAC to check that your reconstruction is correct and the message isn't stale.
If you do this, it's a good idea to have the devices regularly send synchronization messages that contain the full timestamp / message number. This allows them to recover e.g. from prolonged periods of message loss causing the truncated counter to wrap around. For schemes based on sequential message numbering, a typical synchronization message would include both the highest message number sent by the device so far as well as the lowest number they'll accept in return.
To guard against unexpected power loss, the message numbers should be regularly written to permanent storage, such as flash memory. Since you probably don't want to do this after every message, a common solution is to only save the number every, say, 1000 messages, and to add a safety margin of 1000 to the saved value (for the outgoing messages). You should also design your data storage patterns to avoid directly overwriting old data, both to minimize wear on the memory and to avoid data corruption if power is lost during a write. The details of this, however, are a bit outside the scope of this answer.
Ps. Of course, the MAC calculation should also always include the identities of the sender and the intended recipient, so that an attacker can't trick the devices by e.g. echoing a message back to its sender.
This question has always troubled me.
On Linux, when asked for a password, if your input is the correct one, it checks right away, with almost no delay. But, on the other hand, if you type the wrong password, it takes longer to check. Why is that?
I observed this in all Linux distributions I've ever tried.
It's actually to prevent brute force attacks from trying millions of passwords per second. The idea is to limit how fast passwords can be checked and there are a number of rules that should be followed.
A successful user/password pair should succeed immediately.
There should be no discernible difference in reasons for failure that can be detected.
That last one is particularly important. It means no helpful messages like:
Your user name is correct but your password is wrong, please try again
or:
Sorry, password wasn't long enough
Not even a time difference in response between the "invalid user and password" and "valid user but invalid password" failure reasons.
Every failure should deliver exactly the same information, textual and otherwise.
Some systems take it even further, increasing the delay with each failure, or only allowing three failures then having a massive delay before allowing a retry.
This makes it take longer to guess passwords.
I am not sure, but it is quite common to integrate a delay after entering a wrong password to make attacks harder. This makes a attack practicaly infeasible, because it will take you a long time to check only a few passwords.
Even trying a few passwords - birthdates, the name of the cat, and things like that - is turned into no fun.
Basically to mitigate against brute force and dictionary attacks.
From The Linux-PAM Application Developer's Guide:
Planning for delays
extern int pam_fail_delay(pam_handle_t *pamh, unsigned int micro_sec);
This function is offered by Linux-PAM
to facilitate time delays following a
failed call to pam_authenticate() and
before control is returned to the
application. When using this function
the application programmer should
check if it is available with,
#ifdef PAM_FAIL_DELAY
....
#endif /* PAM_FAIL_DELAY */
Generally, an application requests
that a user is authenticated by
Linux-PAM through a call to
pam_authenticate() or pam_chauthtok().
These functions call each of the
stacked authentication modules listed
in the relevant Linux-PAM
configuration file. As directed by
this file, one of more of the modules
may fail causing the pam_...() call to
return an error. It is desirable for
there to also be a pause before the
application continues. The principal
reason for such a delay is security: a
delay acts to discourage brute force
dictionary attacks primarily, but also
helps hinder timed (covert channel)
attacks.
It's a very simple, virtually effortless way to greatly increase security. Consider:
System A has no delay. An attacker has a program that creates username/password combinations. At a rate of thousands of attempts per minute, it takes only a few hours to try every combination and record all successful logins.
System B generates a 5-second delay after each incorrect guess. The attacker's efficiency has been reduced to 12 attempts per minute, effectively crippling the brute-force attack. Instead of hours, it can take months to find a valid login. If hackers were that patient, they'd go legit. :-)
Failed authentification delays are there to reduce the rate of login attempt. The idea that if somebody is trying a dictionary or a brute force attack against one or may user accounts that attacker will be required to wait the fail delay and thus forcing him to take more time and giving you more chance to detect it.
You might also be interested in knowing that, depending on what you are using as a login shell there is usually a way to configure this delay.
In GDM, the delay is set in the gdm.conf file (usually in /etc/gdm/gdm.conf). you need to set RetryDelay=x where x is a value in seconds.
Most linux distribution these day also support having FAIL_DELAY defined in /etc/login.defs allowing you to set a wait time after a failed login attempt.
Finally, PAM also allows you to set a nodelay attribute on your auth line to bypass the fail delay. (Here's an article on PAM and linux)
I don't see that it can be as simple as the responses suggest.
If response to a correct password is (some value of) immediate, don't you only have to wait until longer than that value to know the password is wrong? (at least know probabilistically, which is fine for cracking purposes) And anyway you'd be running this attack in parallel... is this all one big DoS welcome mat?
What I tried before appeared to work, but actually did not; if you care you must review the wiki edit history...
What does work (for me) is, to both lower the value of pam_faildelay.so delay=X in /etc/pam.d/login (I lowered it to 500000, half a second), and also add nodelay (preceded by a space) to the end of the line in common-auth, as described by Gabriel in his answer.
auth [success=1 default=ignore] pam_unix.so nullok_secure nodelay
At least for me (debian sid), only making one of these changes will not shorten the delay appreciably below the default 3 seconds, although it is possible to lengthen the delay by only changing the value in /etc/pam.d/login.
This kind of crap is enough to make a grown man cry!
On Ubuntu 9.10, and I think new versions too, the file you're looking for is located on
/etc/pam.d/login
edit the line:
auth optional pam_faildelay.so delay=3000000
changing the number 3 with another you may want.
Note that to have a 'nodelay' authentication, I THINK you should edit the file
/etc/pam.d/common-auth
too. On the line:
auth [success=1 default=ignore] pam_unix.so nullok_secure
add 'nodelay' to the final (without quotes).
But this final explanation about the 'nodelay' is what I think.
I would like to add a note from a developers perspective. Though this wouldn't be obvious to the naked eye a smart developer would break out of a match query when the match is found. In witness, a successful match would complete faster than a failed match. Because, the matching function would compare the credentials to all known accounts until it finds the correct match. In other words, let's say there are 1,000,000 user accounts in order by IDs; 001, 002, 003 and so on. Your ID is 43,001. So, when you put in a correct username and password, the scan stops at 43,001 and logs you in. If your credentials are incorrect then it scans all 1,000,000 records. The difference in processing time on a dual core server might be in the milliseconds. On Windows Vista with 5 user accounts it would be in the nanoseconds.
I agree. This is an arbitrary programming decision. Putting the delay to one second instead of three doesn't really hurt the crackability of the password, but makes it more user-friendly.
Technically, this deliberate delay is to prevent attacks like the "Linearization attack" (there are other attacks and reasons as well).
To illustrate the attack, consider a program (without this
deliberate delay), which checks an entered serial to see whether it
matches the correct serial, which in this case happens to be
"xyba". For efficiency, the programmer decided to check one
character at a time and to exit as soon as an incorrect character is
found, before beginning the lengths are also checked.
The correct serial length will take longer to process than an incorrect serial length. Even better (for attacker), a serial number
that has the first character correct will take longer than any that
has an incorrect first character. The successive steps in waiting time
is because each time there's one more loop, comparison to go through
on correct input.
So, attacker can select a four-character string and that the string beginning with x takes the most time. (by guess work)
Attacker can then fix character as x and vary the second character, in which case they will find that y takes the longest.
Attacker can then fix the first two characters as xy and vary the third character, in which case they will find that b takes the
longest.
Attacker can then fix the first three character as xyb and vary the fourth character,in which case they will find that a takes the
longest.
Hence, the attackers can recover the serial one character at a time.
Linearization.java.
Linearization.docx, sample output
The serial number is four characters long ans each character has 128
possible values. Then there are 1284 = 228 =
268,435,456 possible serials. If attacker must randomly guess
complete serial numbers, she would guess the serial number in about
227 = 134,217,728 tries, which is an enormous amount of work. On the other hand, by using the linearization attack above, an
average of only 128/2 = 64 guesses are required for each letter, for a
total expected work of about 4 * 64 = 28 = 256 guesses,
which is a trivial amount of work.
Much of the written martial is adapted from this (taken from Mark Stamp's "Information Security: Principles and Practice"). Also the calculations above do not take into account the amount of guesswork needed to to figure out the correct serial length.