Cycript is a console based application that is a blend of Objective-C and JavaScript. Cycript is very useful for dynamic analysis of iOS applications.
If you write any methods or the complete ipa with Swift is it still possible to hook the application on a jailbroken device? Or is Swift safe like "native C" Code on iOS ?
I'm not really familiar with Cycript but I have a little understanding of the Swift compiler.
Swift code will be more resistant to hooking but it should not be completely impossible. NSObject subclasses and Swift classes that are declared #objc should be as accessible as Objective-C code. Pure Swift code, especially in optimised builds would be harder to inject code into because they are often statically dispatched and in many cases will actually be inlined into the calling code.
Where code hasn't been inlined it may may be possible to patch the functions in memory themselves to jump to an alternative function but it wouldn't be as easy as just modifying function tables.
Where key functions have been inlined it may be possible to find and modify each usage if common patterns of code that could be identified and if the function is long enough it may be posible to patch in a jump to an alternate version but this would get really quite tricky.
Related
I don't understand why PowerMock use javassist library and Mockito is not.
Most of the conversations about code generation libraries in Java circle around three libraries: cglib, javassist, and ByteBuddy. Mockito was formerly on cglib, but now uses ByteBuddy as its default code generator.
As ByteBuddy author Rafael Winterhalter notes here:
javassist offers an API for modifying classes and not only for subclassing them. These APIs allow also for byte code-level manipulation while cglib only allows for several hardcoded interceptions.
Though I am not a contributor on any of these mocking frameworks or libraries, it's worth noting that Powermock works in part by editing class implementations to intercept calls to private, static, and final methods and classes within compiled bytecode. This likely explains the requirement to use javassist from Powermock: cglib was not capable of editing existing classes. Mockito, in contrast, needs simpler code generation in order to generate a subclass of the given class; this is functionality that cglib and ByteBuddy were written to provide.
Note that open Powermock issue 727 tracks an incomplete migration of Powermock from Javassist to ByteBuddy.
Now, the reverse: Why doesn't Mockito switch to Javassist instead of ByteBuddy? Again, we don't have a direct answer, but the ByteBuddy tutorial expresses an opinion (under "General Information" for Javassist, emphasis mine):
This library comes with a compiler that takes strings containing Java source code which are translated into Java byte code during the runtime of an application. This is very ambitious and in principle a great idea since Java source code is obviously a great way for describing Java classes. However, the Javassist compiler does not compare to the javac compiler in its functionality and allows for easy mistakes when dynamically composing strings to implement more complex logic. Additionally, Javassist comes with a proxy library which is similar to the JCL's proxy utilities but allows extending classes and is not limited to interfaces. The scope of Javassist's proxy tools remain however equally limited in its API and functionality.
In short: There is anecdotal reason to doubt Javassist's safety/functionality/stability. Mockito did not require Javassist's features, so it could migrate straight from cglib to ByteBuddy. PowerMock did require Javassist's features, and the efforts to migrate PowerMock to ByteBuddy are stalled and ongoing.
Background:
I'm always searching for a language to replace Java for game development. Kotlin looks promising with a good IDE support and Java interop. But one of the FPS killers for a game (on Android especially) is GC usage. So, some libraries (like libgdx) are using pools of objects, custom collections and other tricks to avoid frequent GC run. For Java that can be done in a clear way. Some other JVM languages espesially with functional support using a lot of GC by it's nature, so it is hard to avoid.
Questions:
Does Kotlin creates any invisible GC overhead in comparison to Java?
Which features of Kotlin is better to avoid to have less GC work?
You can write Kotlin Code for the JVM which causes the same allocations than the Java corresponding logic. In both cases you have to carefully check if a library call allocates new memory on the heap, or not. Using Kotlin in combination with LibGDX doesn't introduce any invisible GC overhead. It's an effective way and works well (especially with the ktx extension.
But there are Kotlin language features which may help you to write your code with fewer allocations.
Singletons are a language feature. (Object declarations, companion object )
You can create wrapper classes for primitive types which compile to primitives. But you get the power of type safety and rich domain models (Inline classes).
With the combination of Operator overloading and Inline Functions you can build nice APIs which modify objects without allocating new ones. (Example: Allocation-free Vectorial operations using custom operators)
If you use any kind of dependency injection mechanism or object pooling to connect your game logic and reuse objects, then Reified type parameters may help to use it in a very elegant an short way. You can skip a class as type parameter, if the compiler knows the actual type.
But there is also another option which indeed gives you a different behavior in memory management. Thanks to Kotlin Multiplatform, you can write your game logic as Kotlin common module and cross compile it to native code or to Javascript.
I did this in a sample Game project Candy Crush Clone. It works with Korge a Modern Multiplatform Game Engine for Kotlin. The game runs on the JVM, as HTML web app and as Native binary in Win, Linux, Mac, Android or IOS.
The native compiled code has its own simpler garbage collection and can run faster. So the speed-increase and the different memory management may give you the power reserve to bother even less with the GC.
In conclusion I can recommend Kotlin for Game dev, also for GC critical scenarios. In my projects I tend to create more classes and allocate more memory when I write Kotlin code. But this is a question of programming style, not a technical one.
As a rule of thumb, Kotlin generates bytecode as close as possible to the one generated by Java. So, for example, if you use a function as a value, an inner class will be created, like in Java, but no more. There are also some optimization tricks like IntArray and inline to perform even better.
And as #Peter-Lawrey said, it's always a better idea to measure the values for your specific case.
Technically, your questions comparing Kotlin to Java are moot, they will perform the same. But Kotlin will be a better development experience.
If Java is good for writing Games, then Kotlin would only better due to developer productivity.
Note: the gaming library LWJGL 3 uses Kotlin in part, with GitHub stats showing 67.3% of the code being Kotlin (template module looks to be mostly Kotlin). So asking people who work with LWJGL will give you the best answer to this question since they have a lot of experience in this area.
There are some frameworks out there for dynamic bytecode generation, manipulation and weaving (BCEL, CGLIB, javassist, ASM, MPS). I want to learn about them, but since I don't have much time to know all the details about all of them, I would like to see a sort of comparison chart saying the advantages and disadvantages of one versus the others and an explanation of why.
Here in SO, I found a lot of questions asking something similar, and the answers normally said "you can use cglib or ASM", or "javassist is better than cglib", or "BCEL is old and is dying" or "ASM is the best because it gives X and Y". These answers are useful, but does not fully answer the question in the scope that I want, comparing them more deeply and giving the advantages and disadvantages of each one.
Analysis of bytecode libraries
As I can tell from the answers you got here and ones in the questions that you have looked at, these answers do not formally address the question in the explicit manner you have stated. You asked for a comparison, meanwhile these answers have vaguely stated what one might want based on what your target is (e.g. Do you need to know bytecode? [y/n]), or are too narrow.
This answer is a short analysis of each bytecode framework, and provides a quick comparison at the end.
Javassist
Tiny (javassist.jar (3.21.0) is ~707KB / javassist-rel_3_22_0_cr1.zip is ~1.5MB)
High(/Low)-level
Straightforward
Feature-complete
Requires minimal to no class file format knowledge
Requires moderate Java instruction set knowledge
Minimal learning effort
Has some quirks in the single-line/multi-line compile-and-insert-bytecode methods
I personally prefer Javassist simply because of how quickly you can get to using it and building and manipulating classes with it. The tutorial is straightforward and easy to follow. The jar file is a tiny 707KB, so it is nice and portable; makes it suitable for standalone applications.
ASM
Large (asm-6.0_ALPHA-bin.zip is ~2.9MB / asm-svn-latest.tar.gz (10/15/2016) is ~41MB)
Low(/High)-level
Comprehensive
Feature-complete
Recommend a proficient knowledge of class file format
Requires proficiency with Java instruction set
Moderate learning effort (somewhat complex)
ASM by ObjectWeb is a very comprehensive library which lacks nothing related to building, generating, and loading classes. In fact, it even has class analysis tools with predefined analyzers. It is said to be the industry standard for bytecode manipulation. It is also the reason why I steer clear away from it.
When I see examples of ASM, it seems like a cumbersome beast of a task with the number of lines it takes to modify or load a class. Even some of the parameters to some methods seem a bit cryptic and out of place for Java. With things like ACC_PUBLIC, and plenty of method calls with null everywhere, it honestly does look like it is better suited for a low-level language like C. Why not simply just pass a String literal like "public", or an enum Modifier.PUBLIC? It's more friendly and easy to use. That is my opinion, however.
For reference, here is an ASM (4.0) tutorial: https://www.javacodegeeks.com/2012/02/manipulating-java-class-files-with-asm.html
BCEL
Small (bcel-6.0-bin.zip is 7.3MB / bcel-6.0-src.zip is 1.4MB)
Low-level
Adequate
Gets the job done
Requires proficiency with Java instruction set
Easy to learn
From what I have seen, this library is your basic class library that lets you do everything you need to—if you can spare a few months or years.
Here is a BCEL tutorial that really spells it out: http://www.geekyarticles.com/2011/08/manipulating-java-class-files-with-bcel.html?m=1
cglib
Very tiny (cglib-3.2.5.jar is 295KB/source code)
Depends on ASM
High-level
Feature-complete (Bytecode Generation)
Little or no Java bytecode knowledge needed
Easy to learn
Esoteric Library
Despite the fact that you can read information from classes, and that you can transform classes, the library seems tailored to proxies. The tutorial is all about beans for the proxies, and it even mentions it is used by "data access frameworks to generate dynamic proxy objects and intercept field access." Still, I see no reason why you can't use it for the more simple purpose of bytecode manipulation instead of proxies.
ByteBuddy
Small bin/"Huge" src (by comparison) (byte-buddy-dep-1.8.12.jar is ~2.72 MB / 1.8.12 (zip) is 124.537 MB (exact))
Depends on ASM
High-level
Feature-complete
Personally, a peculiar name for a Service Pattern class (ByteBuddy.class)
Little or no Java byte code knowledge needed
Easy to learn
Long story short, where BCEL is lacking, ByteBuddy is abundant. It uses a primary class called ByteBuddy using the Service Design Pattern. You create a new instance of ByteBuddy, and this represents a class that you want to modify. When you are done with your modifications, you can then make a DynamicType with make().
On their website is a full tutorial with API documentation. The purpose seems to be for rather high-level modifications. When it comes to methods, there does not appear to be anything in the official tutorial, or any 3rd party tutorial, about creating a method from scratch, apart from delegating a method (EDITME if you know where this is explained).
Their tutorial can be found here on their website. Some examples can be found here.
Java Class Assistant (jCLA)
I have my own bytecode library that I am building, which will be called Java Class Assistant, or jCLA for short, because of another project I am working on and because of said quirks with Javassist, but I will not be releasing it to GitHub until it is finished but the project is currently available to browse on GitHub and give feedback on as it is currently in alpha, but still workable enough to be a basic class library (currently working on the compilers; please help me if you can! It will be released a lot sooner!).
It will be quite bare bones with the ability to read and write class files to and from a JAR file, as well as the ability to compile and decompile bytecode to and from source code and class files.
The overall usage pattern makes it rather easy to work with jCLA, though it may take some getting used to and is apparently quite similar to ByteBuddy in its style of methods and method parameters for class modifications:
import jcla.ClassPool;
import jcla.ClassBuilder;
import jcla.ClassDefinition;
import jcla.MethodBuilder;
import jcla.FieldBuilder;
import jcla.jar.JavaArchive;
import jcla.classfile.ClassFile;
import jcla.io.ClassFileOutputStream;
public class JCLADemo {
public static void main(String... args) {
// get the class pool for this JVM instance
ClassPool classes = ClassPool.getLocal();
// get a class that is loaded in the JVM
ClassDefinition classDefinition = classes.get("my.package.MyNumberPrinter");
// create a class builder to modify the class
ClassBuilder clMyNumberPrinter= new ClassBuilder(classDefinition);
// create a new method with name printNumber
MethodBuilder printNumber = new MethodBuilder("printNumber");
// add access modifiers (use modifiers() for convenience)
printNumber.modifier(Modifier.PUBLIC);
// set return type (void)
printNumber.returns("void");
// add a parameter (use parameters() for convenience)
printNumber.parameter("int", "number");
// set the body of the method (compiled to bytecode)
// use body(byte[]) or insert(byte[]) for bytecode
// insert(String) also compiles to bytecode
printNumber.body("System.out.println(\"the number is: \" + number\");");
// add the method to the class
// you can use method(MethodDefinition) or method(MethodBuilder)
clMyNumberPrinter.method(printNumber.build());
// add a field to the class
FieldBuilder HELLO = new FieldBuilder("HELLO");
// set the modifiers for hello; convenience method example
HELLO.modifiers(Modifier.PRIVATE, Modifier.STATIC, Modifier.FINAL);
// set the type of this field
HELLO.type("java.lang.String");
// set the actual value of this field
// this overloaded method expects a VariableInitializer production
HELLO.value("\"Hello from \" + getClass().getSimpleName() + \"!\"");
// add the field to the class (same overloads as clMyNumberPrinter.method())
clMyNumberPrinter.field(HELLO.build());
// redefine
classDefinition = clMyNumberPrinter.build();
// update the class definition in the JVM's ClassPool
// (this updates the actual JVM's loaded class)
classes.update(classDefinition);
// write to disk
JavaArchive archive = new JavaArchive("myjar.jar");
ClassFile classFile = new ClassFile(classDefinition);
ClassFileOutputStream stream = new ClassFileOutputStream(archive);
try {
stream.write(classFile);
} catch(IOException e) {
// print to System.out
} finally {
stream.close();
}
}
}
(VariableInitializer production specification for your convenience.)
As may be implied from the above snippet, each ClassDefinition is immutable. This makes jCLA more secure, thread-safe, network-safe, and easy to use. The system revolves primarily around ClassDefinitions as the object of choice for querying information about a class in a high-level manner, and the system is built in such a way that ClassDefinition is converted to and from target types such as ClassBuilder and ClassFile.
jCLA uses a tiered system for class data. At the bottom, you have the immutable ClassFile: a struct or software representation of a class file. Then you have immutable ClassDefinitions which are converted from ClassFiles into something less cryptic and more manageable and useful to the programmer who is modifying or reading data from the class, and is comparable to information accessed through java.lang.Class. Finally, you have mutable ClassBuilders. The ClassBuilder is how classes are modified or created. It allows that you can create a ClassDefinition directly from the builder from its current state. Creating a new builder for each class is not necessary as the reset() method will clear the variables.
(Analysis of this library will be available as soon as it is ready for release.)
But until then, as of today:
Small (src: 227.704 KB exact, 6/2/2018)
Self-sufficient (no dependencies except Java's shipped library)
High-level
No required knowledge of java bytecode or class files (for tier 1 API, e.g. ClassBuilder, ClassDefinition, etc.)
Easy to learn (even easier if coming from ByteBuddy)
I still recommend learning about java bytecode however. It will make debugging easier.
Comparison
Considering all of these analyses (excluding jCLA for now), the broadest framework is ASM, the easiest to use is Javassist, the most basic implementation is BCEL, and the most high-level for bytecode generation and proxies is cglib.
ByteBuddy deserves its own explanation. It is easy to use like Javassist, but appears to be lacking some of the features that make Javassist great, such as method creation from scratch, so you would need to use ASM for that apparently. If you need to do some lightweight modification with classes, ByteBuddy is the way to go, but for more advanced modification of classes while maintaining a high level of abstraction, Javassist is a better choice.
Note: if I missed a Library, please edit this answer or mention it in a comment.
If your interest in bytecode generation is only to use it, the comparison chart becomes rather simple :
Do you need to understand bytecode?
for javassist : no
for all others : yes
Of course, even with javassist you may be confronted with bytecode concepts at some point. Likewise, some of the other libraries (such as ASM) have a more high-level api and/or tool support to shield you from many of the bytecode details.
What really distinguishes javassist though, is the inclusion of a basic java compiler. This makes it very easy to write complex class transformations : you only have to put a java fragment in a String and use the library to insert it at specific points in the program. The included compiler will build the equivalent bytecode, which is then inserted into the existing classes.
First of all it all depends on your task. Do you want to generate the new code or analyze existing bytecode and how complex analysis you may need. Also how much time you want to invest into learning Java bytecode. You can break down bytecode framework into ones that provide a high level API, that allows you to get away from learning low level opcodes and JVM internals (e.g, javaassist and CGLIB) and low level frameworks when you need to understand JVM or use some bytecode generation tools (ASM and BCEL). For analyzis BCEL historically evolved a bit more, but ASM provides a decent functionality that is easy to extend. Also note, ASM is probably the only framework that provides the most advanced support for STACK_MAP information required by the new bytecode verifier enabled by default in Java 7.
First, I have to say that I really like Groovy and all the good stuff it is bringing to the Java dev world. But since I'm using it for more than little scripts, I have some concerns.
In this Groovy help page about dynamic vs static typing, there is this statement about the absence of compilation error/warning when you have typo in your code because it could be a call to a method added later at runtime:
It might be scary to do away with all of your static typing and
compile time checking at first. But many Groovy veterans will attest
that it makes the code cleaner, easier to refactor, and, well, more
dynamic.
I'm pretty agree with the 'more dynamic' part, but not with cleaner and easier to refactor:
For the other two statements I'm not sure: from my Groovy beginner perspective, this is resulting in less code, but in more difficult to read later and in more trouble to maintain (can not rely on the IDE anymore to find who is declaring a dynamic method and who is using one).
To clarify, I find that reading groovy code is very pleasant, I love the collection and closure (concise and expressive way of tackle complicated problem).
But I have a lot of trouble in these situations:
no more auto-completion inside 'builder' using Map (Of Map (of Map))
everywhere
confusing dynamic methods call (you don't know if it is a typo or a
dynamic name)
method extraction is more complicated inside closure (often resulting in code duplicate: 'it is only a small closure after all')
hard to guess closure parameters when you have to write one for a method of a subsystem
no more learning by browsing the code: you have to use text search instead
I can only saw some benefits with GORM, but in this case the dynamic method are wellknown and my IDE is aware of them (so it is more looking like a systematic code generation than dynamic method for me)
I would be very glad to learn from groovy veteran how they can attest of these benefits.
It does lead to different classes of bugs and processes. It also makes writing tests faster and more natural, helping to alleviate the bug issues.
Discovering where behavior is defined, and used, can be problematic. There isn't a great way around it, although IDEs are getting better at it over time.
Your code shouldn't be more difficult to read--mainline code should be easier to read. The dynamic behavior should disappear into the application, and be documented appropriately for developers that need to understand functionality at those levels.
Magic does make discovery more difficult. This implies that other means of documentation, particularly human-readable tests (think easyb, spock, etc.) and prose, become that much more important.
This is somewhat old, but i'd like to share my experience if someone comes looking for some thoughts on the topic:
Right now we are using eclipse 3.7 and groovy-eclipse 2.7 on a small team (3 developers) and since we don't have tests scripts, mostly of our groovy development we do by explicitly using types.
For example, when using service classes methods:
void validate(Product product) {
// groovy stuff
}
Box pack(List<Product> products) {
def box = new Box()
box.value = products.inject(0) { total, item ->
// some BigDecimal calculations =)
}
box
}
We usually fill out the type, which enable eclipse to autocomplete and, most important, allows us to refactor code, find usages, etc..
This blocks us from using metaprogramming, except for Categories which i found that are supported and is detected by groovy-eclipse.
Still, Groovy is pretty good and a LOT of our business logic is in groovy code.
We had two issues in production code when using groovy, and both cases were due bad manual testing.
We also have a lot of XML building and parsing, and we validate it before sending it to webservices and the likes.
There's a small script we use to connect to an internal system whose usage is very restricted (and not needed in other parts of the system). This code i developed using entirely dynamic typing, overriding methods using metaclass and all that stuff, but this is an exception.
I think groovy 2.0 (with groovy-eclipse coming along, of course) and it's #TypeChecked will be great for those of us that uses groovy as a "java++".
To me there are 2 types of refactoring:
IDE based refactoring (extract to method, rename method, introduce variable, etc.).
Manual refactoring. (moving a method to a different class, changing the return value of a method)
For IDE based refactoring I haven't found an IDE that does as good of a job with Groovy as it does with Java. For example in eclipse when you extract to method it looks for duplicate instances to refactor to call the method instead of having duplicated code. For Groovy, that doesn't seem to happen.
Manual refactoring is where I believe that you could see refactoring made easier. Without tests though I would agree that it is probably harder.
The statement at cleaner code is 100% accurate. I would venture a guess that good Java to good Groovy code is at least a 3:1 reduction in lines of code. Being a newbie at Groovy though I would strive to learn at least 1 new way to do something everyday. Something that greatly helped me improve my Groovy was to simply read the APIs. I feel that Collection, String, and List are probably the ones that have the most functionality and I used the most to help make my Groovy code actually Groovy.
http://groovy.codehaus.org/groovy-jdk/java/util/Collection.html
http://groovy.codehaus.org/groovy-jdk/java/lang/String.html
http://groovy.codehaus.org/groovy-jdk/java/util/List.html
Since you edited the question I'll edit my answer :)
One thing you can do is tell intellij about the dynamic methods on your objects: What does 'add dynamic method' do in Groovy/IntelliJ?. That might help a little bit.
Another trick that I use is to type my objects when doing the initial coding and remove the typing when I'm done. For example I can never seem to remember if it's .substring(..) or .subString(..) on a String. So if you type your object you get a little better code completion.
As for your other bullet points, I'd really need to look at some code to be able to give a better answer.
I'm looking for a way to run an arbitrary Haskell code safely (or refuse to run unsafe code).
Must have:
module/function whitelist
timeout on execution
memory usage restriction
Capabilities I would like to see:
ability to kill thread
compiling the modules to native code
caching of compiled code
running several interpreters concurrently
complex datatype for compiler errors (insted of simple message in String)
With that sort of functionality it would be possible to implement a browser plugin capable of running arbitrary Haskell code, which is the idea I have in mind.
EDIT: I've got two answers, both great. Thanks! The sad part is that there doesn't seem to be ready-to-go library, just a similar program. It's a useful resource though. Anyway I think I'll wait for 7.2.1 to be released and try to use SafeHaskell in my own program.
We've been doing this for about 8 years now in lambdabot, which supports:
a controlled namespace
OS-enforced timeouts
native code modules
caching
concurrent interactive top-levels
custom error message returns.
This series of rules is documented, see:
Safely running untrusted Haskell code
mueval, an alternative implementation based on ghc-api
The approach to safety taken in lambdabot inspired the Safe Haskell language extension work.
For approaches to dynamic extension of compiled Haskell applications, in Haskell, see the two papers:
Dynamic Extension of Typed Functional Languages, and
Dynamic applications from the ground up.
GHC 7.2.1 will likely have a new facility called SafeHaskell which covers some of what you want. SafeHaskell ensures type-safety (so things like unsafePerformIO are outlawed), and establishes a trust mechanism, so that a library with a safe API but implemented using unsafe features can be trusted. It is designed exactly for running untrusted code.
For the other practical aspects (timeouts and so on), lambdabot as Don says would be a great place to look.