How to fix "Path Manipulation Vulnerability" in some Java Code? - security

The below simple java code getting Fortify Path Manipulation error. Please help me to resolve this. I am struggling from long time.
public class Test {
public static void main(String[] args) {
File file=new File(args[0]);
}
}

Try to normalize the URL before using it
https://docs.oracle.com/javase/7/docs/api/java/net/URI.html#normalize()
Path path = Paths.get("/foo/../bar/../baz").normalize();
or use normalize from org.apache.commons.io.FilenameUtils
https://commons.apache.org/proper/commons-io/javadocs/api-1.4/org/apache/commons/io/FilenameUtils.html#normalize(java.lang.String)
Stirng path = FilenameUtils.normalize("/foo/../bar/../baz");
For both the result will be \baz

Looking at the OWASP page for Path Manipulation, it says
An attacker can specify a path used in an operation on the filesystem
You are opening a file as defined by a user-given input. Your code is almost a perfect example of the vulnerability! Either
Don't use the above code (don't let the user specify the input file as an argument)
Let the user choose from a list of files that you supply (an array of files with an integer choice)
Don't let the user supply the filename at all, remove the configurability
Accept the vulnerability but protect against it by checking the filename (although this is the worst thing to do - someone may get round it anyway).
Or re-think your application's design.

Fortify will flag the code even if the path/file doesn't come from user input like a property file. The best way to handle these is to canonicalize the path first, then validate it against a white list of allowed paths.
Bad:
public class Test {
public static void main(String[] args) {
File file=new File(args[0]);
}
}
Good:
public class Test {
public static void main(String[] args) {
File file=new File(args[0]);
if (!isInSecureDir(file)) {
throw new IllegalArgumentException();
}
String canonicalPath = file.getCanonicalPath();
if (!canonicalPath.equals("/img/java/file1.txt") &&
!canonicalPath.equals("/img/java/file2.txt")) {
// Invalid file; handle error
}
FileInputStream fis = new FileInputStream(f);
}
Source: https://www.securecoding.cert.org/confluence/display/java/FIO16-J.+Canonicalize+path+names+before+validating+them

Only allow alnum and a period in input. That means you filter out the control chars, "..", "/", "\" which would make your files vulnerable. For example, one should not be able to enter /path/password.txt.
Once done, rescan and then run Fortify AWB.

I have a solution to the Fortify Path Manipulation issues.
What it is complaining about is that if you take data from an external source, then an attacker can use that source to manipulate your path. Thus, enabling the attacker do delete files or otherwise compromise your system.
The suggested remedy to this problem is to use a whitelist of trusted directories as valid inputs; and, reject everything else.
This solution is not always viable in a production environment. So, I suggest an alternative solution. Parse the input for a whitelist of acceptable characters. Reject from the input, any character you don't want in the path. It could be either removed or replaced.
Below is an example. This does pass the Fortify review. It is important to remember here to return the literal and not the char being checked. Fortify keeps track of the parts that came from the original input. If you use any of the original input, you may still get the error.
public class CleanPath {
public static String cleanString(String aString) {
if (aString == null) return null;
String cleanString = "";
for (int i = 0; i < aString.length(); ++i) {
cleanString += cleanChar(aString.charAt(i));
}
return cleanString;
}
private static char cleanChar(char aChar) {
// 0 - 9
for (int i = 48; i < 58; ++i) {
if (aChar == i) return (char) i;
}
// 'A' - 'Z'
for (int i = 65; i < 91; ++i) {
if (aChar == i) return (char) i;
}
// 'a' - 'z'
for (int i = 97; i < 123; ++i) {
if (aChar == i) return (char) i;
}
// other valid characters
switch (aChar) {
case '/':
return '/';
case '.':
return '.';
case '-':
return '-';
case '_':
return '_';
case ' ':
return ' ';
}
return '%';
}
}

Assuming you're running Fortify against a web application, during your triage of Fortify vulnerabilities that would likely get marked as "Not an issue". Reasoning being A) obviously this is test code and B) unless you have multiple personality disorder you're not going to be doing a path manipulation exploit against your self when you run that test app.
If very common to see little test utilities committed to a repository which produces this style of false positive.
As for your compilation errors, that generally comes down to classpath issues.

We have code like below which was raising Path Manipulation high category issue in fortify .
String.join(delimeter,string1,string2,string2,string4);
Our program is to deal with AWS S3 bucket so, we changed as below and it worked .
com.amazonaws.util.StringUtils.join(delimeter,string1,string2,string2,string4);

Using the Tika library FilenameUtils.normalize solves the fortify issue.
import org.apache.tika.io.FilenameUtils;
public class Test {
public static void main(String[] args) {
String filePath = FilenameUtils.normalize(args[0]); //This line solve issue.
File file=new File(filePath);
}
}

Try this for replacing FileInputStream. You will need to close your project and open again to accurately see whether changes worked.
File to byte[] in Java

Use Normalize() function in C# and it resolved the fortify vulnerability in next scan.
string s = #:c:\temp\scan.log".Normalize();

Use regex to validate the file path and file name
fileName = args[0];
final String regularExpression = "([\\w\\:\\\\w ./-]+\\w+(\\.)?\\w+)";
Pattern pattern = Pattern.compile(regularExpression);
boolean isMatched = pattern.matcher(fileName).matches();

Related

how to check which constraints would be violated by a presumed solution?

In some cases the solver fails to find a solution for my model, which I think is there.
So I would like to populate a solution, and then check which constraint is violated.
How to do that with choco-solver?
Using choco-solver 4.10.6.
Forcing a solution
I ended up adding constraints to force variables to values of my presumed solution:
e.g.
// constraints to force given solution
vehicle2FirstStop[0].eq(model.intVar(4)).post();
vehicle2FirstStop[1].eq(model.intVar(3)).post();
nextStop[1].eq(model.intVar(0)).post();
nextStop[2].eq(model.intVar(1)).post();
...
and then
model.getSolver().showContradiction();
if (model.getSolver().solve()) { ....
Shows the first contradiction of the presumed solution, e.g.
/!\ CONTRADICTION (PropXplusYeqZ(sum_exp_49, mul_exp_51, ...
So the next step is to find out where terms such as sum_exp_49 come from.
Matching the contradiction terms with the code
Here is a simple fix for constraints which will hopefully provide enough information. We can override the post() and associates() methods of model, so that it dumps the java source filename and line number when a constraint is posted/variable is created.
Model model = new Model("Vrp1RpV") {
/**
* retrieve the filename and line number of first caller outside of choco-solver from stacktrace
*/
String getSource() {
String source = null;
StackTraceElement[] stackTraceElements = Thread.currentThread().getStackTrace();
// starts from 3: thread.getStackTrace() + this.getSource() + caller (post() or associates())
for (int i = 3; i < stackTraceElements.length; i++) {
// keep rewinding until we get out of choco-solver packages
if (!stackTraceElements[i].getClassName().toString().startsWith("org.chocosolver")) {
source = stackTraceElements[i].getFileName() + ":" + stackTraceElements[i].getLineNumber();
break;
}
}
return source;
}
#Override
public void post(Constraint... cs) throws SolverException {
String source=getSource();
// dump each constraint along source location
for (Constraint c : cs) {
System.err.println(source + " post: " + c);
}
super.post(cs);
}
#Override
public void associates(Variable variable) {
System.err.println(getSource() + " associates: " + variable.getName());
super.associates(variable);
}
};
This will dump things like:
Vrp1RpV2.java:182 post: ARITHM ([prop(EQ_exp_47.EQ.mul_exp_48)])
Vrp1RpV2.java:182 associates: sum_exp_49
Vrp1RpV2.java:182 post: ARITHM ([prop(mul_exp_48.EQ.sum_exp_49)])
Vrp1RpV2.java:182 associates: EQ_exp_50
Vrp1RpV2.java:182 post: BASIC_REIF ([(stop2vehicle[2] = 1) <=> EQ_exp_50])
...
From there it is possible to see where sum_exp_49 comes from.
EDIT: added associates() thanks to #cprudhom suggestion on https://gitter.im/chocoteam/choco-solver

Spell checker in android

I want to create an application in which it checks if the word typed by user is correct or not using Google Dictionary ?
i have used the below link . But the problem with the given example is that it suggests the different words. I don't want suggestion, instead i want to only check that word entered is correct or not.
I haven't worked on it yet. But you can probably modify it as:
When you get the suggestions, instead of appending them to StringBuilder, and showing that StringBuilder to MainView, just compare all suggestions with your input string of edittext.
If it matches, then the spell is correct, else the spell is incorrect.
Code snippet:
#Override
public void onGetSuggestions(final SuggestionsInfo[] arg0) {
isSpellCorrect = false;
final StringBuilder sb = new StringBuilder();
for (int i = 0; i < arg0.length; ++i) {
// Returned suggestions are contained in SuggestionsInfo
final int len = arg0[i].getSuggestionsCount();
if(editText1.getText().toString().equalsIgnoreCase(arg0[i].getSuggestionAt(j))
{
isSpellCorrect = true;
break;
}
}
}
Hope this helps.

Console application: How to update the display without flicker?

Using C# 4 in a Windows console application that continually reports progress how can I make the "redraw" of the screen more fluid?
I'd like to do one of the following:
- Have it only "redraw" the part of the screen that's changing (the progress portion) and leave the rest as is.
- "Redraw" the whole screen but not have it flicker.
Currently I re-write all the text (application name, etc.). Like this:
Console.Clear();
WriteTitle();
Console.WriteLine();
Console.WriteLine("Deleting:\t{0} of {1} ({2})".FormatString(count.ToString("N0"), total.ToString("N0"), (count / (decimal)total).ToString("P2")));
Which causes a lot of flickering.
Try Console.SetCursorPosition. More details here: How can I update the current line in a C# Windows Console App?
static void Main(string[] args)
{
Console.SetCursorPosition(0, 0);
Console.Write("################################");
for (int row = 1; row < 10; row++)
{
Console.SetCursorPosition(0, row);
Console.Write("# #");
}
Console.SetCursorPosition(0, 10);
Console.Write("################################");
int data = 1;
System.Diagnostics.Stopwatch clock = new System.Diagnostics.Stopwatch();
clock.Start();
while (true)
{
data++;
Console.SetCursorPosition(1, 2);
Console.Write("Current Value: " + data.ToString());
Console.SetCursorPosition(1, 3);
Console.Write("Running Time: " + clock.Elapsed.TotalSeconds.ToString());
Thread.Sleep(1000);
}
Console.ReadKey();
}
I know this question is a bit old but I found if you set Console.CursorVisible = false then the flickering stops as well.
Here's a simple working demo that shows multi-line usage without flickering. It shows the current time and a random string every second.
private static void StatusUpdate()
{
var whiteSpace = new StringBuilder();
whiteSpace.Append(' ', 10);
var random = new Random();
const string chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
var randomWord = new string(Enumerable.Repeat(chars, random.Next(10)).Select(s => s[random.Next(s.Length)]).ToArray());
while (true)
{
Console.SetCursorPosition(0, 0);
var sb = new StringBuilder();
sb.AppendLine($"Program Status:{whiteSpace}");
sb.AppendLine("-------------------------------");
sb.AppendLine($"Last Updated: {DateTime.Now}{whiteSpace}");
sb.AppendLine($"Random Word: {randomWord}{whiteSpace}");
sb.AppendLine("-------------------------------");
Console.Write(sb);
Thread.Sleep(1000);
}
}
The above example assumes your console window is blank to start. If not, make sure to use Console.Clear() first.
Technical Note:
SetCursorPosition(0,0) places the cursor back to the top (0,0) so the next call to Console.Write will start from line 0, char 0. Note, it doesn't delete the previous content before writing. As an example, if you write "asdf" over a previous line such as "0123456", you'll end up with something like "asdf456" on that line. For that reason, we use a whiteSpace variable to ensure any lingering characters from the previous line are overwritten with blank spaces. Adjust the length of the whiteSpace variable to meet your needs. You only need the whiteSpace variable for lines that change.
Personal Note:
For my purposes, I wanted to show the applications current status (once a second) along with a bunch of other status information and I wanted to avoid any annoying flickering that can happen when you use Console.Clear(). In my application, I run my status updates behind a separate thread so it constantly provides updates even though I have numerous other threads and long running tasks going at the same time.
Credits:
Thanks to previous posters and dtb for the random string generator used in the demo.
How can I generate random alphanumeric strings in C#?
You could try to hack something together using the core libraries.
Rather than waste your time for sub-standard results, I would check out this C# port of the ncurses library (which is a library used for formatting console output):
Curses Sharp
I think you can use \r in Windows console to return the beginning of a line.
You could also use SetCursorPosition.
I would recommend the following extension methods. They allow you to use a StringBuilder to refresh the console view without any flicker, and also tidies up any residual characters on each line
The Problem: The following demo demonstrates using a standard StringBuilder, where updating lines that are shorter than the previously written line get jumbled up. It does this by writing a short string, then a long string on a loop:
public static void Main(string[] args)
{
var switchTextLength = false;
while(true)
{
var sb = new StringBuilder();
if (switchTextLength)
sb.AppendLine("Short msg");
else
sb.AppendLine("Longer message");
sb.UpdateConsole();
switchTextLength = !switchTextLength;
Thread.Sleep(500);
}
}
Result:
The Solution: By using the extension method provided below, the issue is resolved
public static void Main(string[] args)
{
var switchTextLength = false;
while(true)
{
var sb = new StringBuilder();
if (switchTextLength)
sb.AppendLineEx("Short msg");
else
sb.AppendLineEx("Longer message");
sb.UpdateConsole();
switchTextLength = !switchTextLength;
Thread.Sleep(500);
}
}
Result:
Extension Methods:
public static class StringBuilderExtensions
{
/// <summary>
/// Allows StrinbBuilder callers to append a line and blank out the remaining characters for the length of the console buffer width
/// </summary>
public static void AppendLineEx(this StringBuilder c, string msg)
{
// Append the actual line
c.Append(msg);
// Add blanking chars for the rest of the buffer
c.Append(' ', Console.BufferWidth - msg.Length - 1);
// Finish the line
c.Append(Environment.NewLine);
}
/// <summary>
/// Combines two StringBuilders using AppendLineEx
/// </summary>
public static void AppendEx(this StringBuilder c, StringBuilder toAdd)
{
foreach (var line in toAdd.ReadLines())
{
c.AppendLineEx(line);
}
}
/// <summary>
/// Hides the console cursor, resets its position and writes out the string builder
/// </summary>
public static void UpdateConsole(this StringBuilder c)
{
// Ensure the cursor is hidden
if (Console.CursorVisible) Console.CursorVisible = false;
// Reset the cursor position to the top of the console and write out the string builder
Console.SetCursorPosition(0, 0);
Console.WriteLine(c);
}
}
I actually had this issue so I made a quick simple method to try and eliminate this.
static void Clear(string text, int x, int y)
{
char[] textChars = text.ToCharArray();
string newText = "";
//Converts the string you just wrote into a blank string
foreach(char c in textChars)
{
text = text.Replace(c, ' ');
}
newText = text;
//Sets the cursor position
Console.SetCursorPosition(x, y);
//Writes the blank string over the old string
Console.WriteLine(newText);
//Resets cursor position
Console.SetCursorPosition(0, 0);
}
It actually worked surprisingly well and I hope it may work for you!
Naive approach but for simple applications is working:
protected string clearBuffer = null; // Clear this if window size changes
protected void ClearConsole()
{
if (clearBuffer == null)
{
var line = "".PadLeft(Console.WindowWidth, ' ');
var lines = new StringBuilder();
for (var i = 0; i < Console.WindowHeight; i++)
{
lines.AppendLine(line);
}
clearBuffer = lines.ToString();
}
Console.SetCursorPosition(0, 0);
Console.Write(clearBuffer);
Console.SetCursorPosition(0, 0);
}
Console.SetCursorPosition(0, 0); //Instead of Console.Clear();
WriteTitle();
Console.WriteLine();
Console.WriteLine("Deleting:\t{0} of {1} ({2})".FormatString(count.ToString("N0")

Tomcat secured static content

I'm making a service that among other has the "photo albums" feature that serve photos to users. User has to be "allowed" to see the photo from the album. So sending the direct link to other person shouldn't allow to view photo.
Photos are stored in the folder outside of the context.
What I need to do is to perform some checks when user requests the photo and then if checks are OK - serve the file. I want to avoid making a wheel and just let tomcat serve the image as it usually does for static files. Can you give some advice on that?
Ok, guys.
After struggling hard with this question I think I've finally found out what to do to solve it. First of all it looks like the question actually decomposes into two independent tasks. One of them is securing access to some resources and second one is feeding resources from the folder outside of the context.
First task is trivial and can be solved by writing a simple filter hanged to "/".
Second task is much less trivial but fortunately also can be resolved. Tomcat uses the implementation of javax.naming.directory.DirContext to load all resources of the given web application including class files. It also allows you to provide the custom implementation of this interface and configure it in the context.xml file. The default implementation is org.apache.naming.resources.FileDirContext. Details here: http://tomcat.apache.org/tomcat-6.0-doc/config/resources.html
I've created my own implementation of DirContext by simply extending FileDirContext. Luckily enough there was a single method that had to be overwritten in order to "hook up" file discovery. The method is called file().
I'm posting my test code here. It is far from perfect and does not take into account the corner cases like renaming files but I don't think that these are needed under a normal run of the server.
The basic idea under this code is to check if the path starts with "virtual directory" prefix and if it is - search for file in the other place in the filesystem (I know there is some duplicate code there but I hope you're not that lazy to remove it if you ever want to use it :-). setVirtualName and setVirtualBase are called automatically to inject the configuration params.
/**
* TODO: add javadocs
*
* #author Juriy Bura
*/
public class VirtualFolderDirContext extends FileDirContext {
private String virtualName;
private String realName;
private File virtualBase;
private String absoluteVirtualBase;
public VirtualFolderDirContext() {
super();
}
public VirtualFolderDirContext(Hashtable env) {
super(env);
}
public void setVirtualName(String path) {
virtualName = path;
}
public void setVirtualBase(String base) {
this.realName = base;
virtualBase = new File(realName);
try {
virtualBase = virtualBase.getCanonicalFile();
} catch (IOException e) {
// Ignore
}
this.absoluteVirtualBase = virtualBase.getAbsolutePath();
}
protected File file(String name) {
File file = null;
boolean virtualFile = name.startsWith(virtualName + "/");
if (virtualFile) {
file = new File(virtualBase, name.substring(virtualName.length()));
} else {
file = new File(base, name);
}
if (file.exists() && file.canRead()) {
if (allowLinking)
return file;
// Check that this file belongs to our root path
String canPath = null;
try {
canPath = file.getCanonicalPath();
} catch (IOException e) {
}
if (canPath == null)
return null;
// Check to see if going outside of the web application root
if (!canPath.startsWith(absoluteBase) && !canPath.startsWith(absoluteVirtualBase)) {
return null;
}
// Case sensitivity check
if (caseSensitive) {
String fileAbsPath = file.getAbsolutePath();
if (fileAbsPath.endsWith("."))
fileAbsPath = fileAbsPath + "/";
String absPath = normalize(fileAbsPath);
if (canPath != null)
canPath = normalize(canPath);
if (virtualFile) {
if ((absoluteVirtualBase.length() < absPath.length())
&& (absoluteVirtualBase.length() < canPath.length())) {
absPath = absPath.substring(absoluteVirtualBase.length() + 1);
if ((canPath == null) || (absPath == null))
return null;
if (absPath.equals(""))
absPath = "/";
canPath = canPath.substring(absoluteVirtualBase.length() + 1);
if (canPath.equals(""))
canPath = "/";
if (!canPath.equals(absPath))
return null;
}
} else {
if ((absoluteBase.length() < absPath.length())
&& (absoluteBase.length() < canPath.length())) {
absPath = absPath.substring(absoluteBase.length() + 1);
if ((canPath == null) || (absPath == null))
return null;
if (absPath.equals(""))
absPath = "/";
canPath = canPath.substring(absoluteBase.length() + 1);
if (canPath.equals(""))
canPath = "/";
if (!canPath.equals(absPath))
return null;
}
}
}
} else {
return null;
}
return file;
}
}
After you have this class in place you have to jar it and put that jar into the Tomcat lib folder. For obvious reasons it cannot go together with war file. In your context.xml you should add a config lines like these:
<?xml version="1.0" encoding="UTF-8"?>
<Context antiResourceLocking="true" antiJARLocking="true">
<Resources
className="com.juriy.tomcat.virtualdir.VirtualFolderDirContext"
virtualName="/upload"
virtualBase="c:/temp/up">
</Resources>
...
...
Now any time user asks for /upload/ it will be resolved to c:\temp. With this technique you can implement loading resources from virtually any location: http, shared folder, database, even version control system. So it is pretty cool.
P.S. I've killed the whole day to make this all work together so don't hesitate to give me your vote if you like the answer :-))
Cheers
Juriy

Is there an additional runtime cost for using named parameters?

Consider the following struct:
public struct vip
{
string email;
string name;
int category;
public vip(string email, int category, string name = "")
{
this.email = email;
this.name = name;
this.category = category;
}
}
Is there a performance difference between the following two calls?
var e = new vip(email: "foo", name: "bar", category: 32);
var e = new vip("foo", 32, "bar");
Is there a difference if there are no optional parameters defined?
I believe none. It's only a language/compiler feature, call it syntactic sugar if you like. The generated CLR code should be the same.
There's a compile-time cost, but not a runtime one...and the compile time is very, very minute.
Like extension methods or auto-implemented properties, this is just magic the compiler does, but in reality generates the same IL we're all familiar with and have been using for years.
Think about it this way, if you're using all the parameters, the compiler would call the method using all of them, if not, it would generate something like this behind the scenes:
var e = new vip(email: "foo", category: 32); //calling
//generated, this is what it's actually saving you from writing
public vip(string email, int category) : this(email, category, "bar") { }
No it is a compile-time feature only. If you inspect the generated IL you'll see no sign of the named parameters. Likewise, optional parameters is also a compile-time feature.
One thing to keep in mind regarding named parameters is that the names are now part of the signature for calling a method (if used obviously) at compile time. I.e. if names change the calling code must be changed as well if you recompile. A deployed assembly, on the other hand, will not be affected until recompiled, as the names are not present in the IL.
There shouldn't be any. Basically, named parameters and optional parameters are syntactic sugar; the compiler writes the actual values or the default values directly into the call site.
EDIT: Note that because they are a compiler feature, this means that changes to the parameters only get updated if you recompile the "clients". So if you change the default value of an optional parameter, for example, you will need to recompile all "clients", or else they will use the old default value.
Actually, there is cost at x64 CLR
Look at here http://www.dotnetperls.com/named-parameters
I am able to reproduce the result: named call takes 4.43 ns, and normal call takes 3.48 ns
(program runs in x64)
However, in x86, both take around 0.32 ns
The code is attached below, compile and run it yourself to see the difference.
Note that in VS2012 the default targat is AnyCPU x86 prefered, you have to switch to x64 to see the difference.
using System;
using System.Diagnostics;
class Program
{
const int _max = 100000000;
static void Main()
{
Method1();
Method2();
var s1 = Stopwatch.StartNew();
for (int i = 0; i < _max; i++)
{
Method1();
}
s1.Stop();
var s2 = Stopwatch.StartNew();
for (int i = 0; i < _max; i++)
{
Method2();
}
s2.Stop();
Console.WriteLine(((double)(s1.Elapsed.TotalMilliseconds * 1000 * 1000) /
_max).ToString("0.00 ns"));
Console.WriteLine(((double)(s2.Elapsed.TotalMilliseconds * 1000 * 1000) /
_max).ToString("0.00 ns"));
Console.Read();
}
static void Method1()
{
Method3(flag: true, size: 1, name: "Perl");
}
static void Method2()
{
Method3(1, "Perl", true);
}
static void Method3(int size, string name, bool flag)
{
if (!flag && size != -1 && name != null)
{
throw new Exception();
}
}
}

Resources