 |
Programmatically Injecting Events on Android – Part 1By Radu Motisan Posted on April 29th, 2012 , 10250 Views (Rate 26.42) |
Note: Part two has been published and is available here.
The successful Android platform has been around for a few years now. End users get lots of bells and whistles in fancy applications, but for developers, this "open-intended" platform seems even more tangled up than ever.
There are various APIs that still do not work, others that get deprecated-and-out in a single release, or some that are locked up for the supreme security purpose, invoked so very often by platform developers, that probably feel they've given the right answer.
All in one, Android also comes with a lot of frustration, because tools that are needed to accomplish various tasks or even more advanced applications, are put behind bars on purpose.
Another example of many is injecting a key press programmatically or the similar counter-part, injecting a touch event (mouse).
Because a malicious programmer might develop a software that would open your Market app and download payed apps without you knowing about it, developers world wide are prohibited from sending a key press programmatically to other applications except their own.
This is just ridiculous narrow thinking.
Of course there will always be a balance between security and functionality, but with paranoia it quickly turns to total failure. In other words why limit something so useful that can create so little damage? Users are warned about risks with normal permissions and it should stay that way for other features as well.
I'll just leave them with their issues and get back to the topic: I am aware of three methods for injecting events programmatically. This refers both to keyboard events (keys) and mouse events (touch events).
Method 1: Using internal APIs
This approach has its risks, like it is always with internal, unpublished APIs.
The idea is to get an instance of WindowManager in order to access the injectKeyEvent / injectPointerEvent methods.
The ServiceManager and WindowsManager are defined as Stubs. We can then bind to these services and call the methods we need. The interfaces are included in the sample code attached at the end of this article.
To send a key do the following:
To send touch/mouse events use:
This works fine, but only inside your application
The moment you're trying to inject keys/touch events to any other window, you'll get a force close because of the following exception:
Not much joy, as INJECT_EVENTS is a system permission. A possible solution is discussed here and here.
Method 2: Using an instrumentation object
This is a clean solution based on public API, but unfortunately it still requires that INJECT_EVENTS permission.
For touch events you can use:
All good inside the test application, and will crash instantly when trying to inject keys to outside apps, not because the approach doesn't work, but because Android Developers have chosen so. Thanks guys, you rock! Not.
By looking at sendPointerSync's code, you will quickly see it uses the same approach as presented in method 1). So this is the same thing, but packed nicely in a easy to use API:
Method 3: Direct event injection to /dev/input/eventX
Linux exposes a uniform input event interface for each device as /dev/input/eventX where X is an integer. We can use it directly and skip the above Android Platform permission issues.
For this to work, we will need root access, so this approach only works on a rooted device.
I felt it was easier to deal with linux using native C code, but a pure java implementation is also possible. Therefore I have added a small JNI component to handle the interface with /dev/input/eventX.
The sample code I wrote doesn't detect the X number automatically, so make sure you set that before running the code. By default I set it to event3. You can change that in NativeInput.java, see code below.
As I said, this last method requires root. By default the eventX files have the permission set for 660 (read and write for Owner and Group only). To inject keys from our application, we need to make it writable. So do this first:
You will need root to run the chmod command.
Sample Code
Other great resources:
Generating keypresses programmatically
Internal input event handling in the Linux kernel and the Android userspace
androidscreencast
Note: Part two has been published and is available here.
|
|
May 30th, 2012 at 10:32 am
Hey Radu
Thx for the nice little demo.
I cant get method 3 to work on my sgs2 with android 4.03. I installed your demo application and chanced it to method 3. When it was installe I did adb shell su chmod 666 /dev/input/event3 but nothing happens when i pres start.
Logcat says:
5-30 10:21:38.642: D/dalvikvm(19464): Late-enabling CheckJNI
05-30 10:21:38.657: I/dalvikvm(19464): Turning on JNI app bug workarounds for target SDK version 5…
05-30 10:21:38.662: D/dalvikvm(19464): Debugger has detached; object registry had 1 entries
05-30 10:21:38.737: D/dalvikvm(19464): Trying to load lib /data/data/net.pocketmagic.keyinjector/lib/libinput.so 0×41507800
05-30 10:21:38.742: D/dalvikvm(19464): Added shared lib /data/data/net.pocketmagic.keyinjector/lib/libinput.so 0×41507800
05-30 10:21:38.747: D/JNI(19464): Debug enabled.
05-30 10:21:38.747: D/JNI(19464): intCreate call (/dev/input/event3)
05-30 10:21:38.747: D/JNI(19464): intCreate success: 44
05-30 10:21:38.902: D/CLIPBOARD(19464): Hide Clipboard dialog at Starting input: finished by someone else… !
05-30 10:21:56.802: D/AKI(19464): Inject method:3
05-30 10:21:56.802: D/JNI(19464): intSendEvent call (44,1,46,1)
05-30 10:21:56.802: D/JNI(19464): intSendEvent done:16
05-30 10:21:56.802: D/JNI(19464): intSendEvent call (44,1,46,0)
05-30 10:21:56.802: D/JNI(19464): intSendEvent done:16
05-30 10:21:57.802: D/AKI(19464): Inject method:3
05-30 10:21:57.802: D/JNI(19464): intSendEvent call (44,1,46,1)
05-30 10:21:57.802: D/JNI(19464): intSendEvent done:16
05-30 10:21:57.802: D/JNI(19464): intSendEvent call (44,1,46,0)
05-30 10:21:57.802: D/JNI(19464): intSendEvent done:16
05-30 10:22:36.947: D/dalvikvm(19464): Debugger has detached; object registry had 1 entries
Can you se what im doing wrong here?
July 28th, 2012 at 7:50 pm
what is the su? Should I enter my computers password?
August 6th, 2012 at 12:35 pm
Hi,
step1 My device(Motorola Xoom 3.2) is rooted.and following below steps to perform keyInject at any Pixel on device
step–1(a) Set event3 chmod to 777
(b) Running below Code in a background thread.
m_ni.SendKey(46 , true);
m_ni.SendKey(46, false);
step–2 To check key Event create another application with no title-bar and having A single button with OnClickListener. when the button is clicked it will display a Toast.
step -3 Launch 1st step App and then 2nd step App
but nothing is happening neither it display any Exception nor it display the toast.
please help!!!!!!
August 9th, 2012 at 7:17 am
Hi Radu,
August 9th, 2012 at 7:21 am
Hi Radu,
What changes I have made to perform touch events on rooted device??
Please reply!!!!
September 5th, 2012 at 11:51 am
[...] is some ways in Android to inject events http://www.pocketmagic.net/?p=2640 In this case can help: 1) Using of internal API. But i cant find API in sources witch i can use. [...]
September 18th, 2012 at 3:10 am
Does method 1 still work?
October 26th, 2012 at 7:16 pm
@kyle, inside your own application yes;
November 8th, 2012 at 5:00 pm
Thanks for a great explanaition + example!
I ran your code and methode 1 and 2 work like you explained, but I can’t get methode 3 to work..
Any tips?
Thanks in advance!
November 8th, 2012 at 5:19 pm
Sosh, did you set the permissions right?
November 28th, 2012 at 5:45 am
@Radu:
in 4.1 4.2
IWindowManager have changed there is no inject* method ever.
But we can use KeyEvent home = new KeyEvent(KeyEvent.ACTION_DOWN,
KeyEvent.KEYCODE_HOME);
InputManager.getInstance().injectInputEvent(home,
InputManager.INJECT_INPUT_EVENT_MODE_WAIT_FOR_FINISH);
November 28th, 2012 at 5:46 am
in 4.1 4.2
we should do like below:
KeyEvent home = new KeyEvent(KeyEvent.ACTION_DOWN,
KeyEvent.KEYCODE_HOME);
InputManager.getInstance().injectInputEvent(home,
InputManager.INJECT_INPUT_EVENT_MODE_WAIT_FOR_FINISH);
November 28th, 2012 at 8:51 am
@insomnia: I didn’t have the chance to test that yet, but it wouldn’t be something new for google to break previous android functionality with each new release.
January 10th, 2013 at 12:55 am
Hello there Radu
First of all congratz on your excellent work on this post and the CursorOverlay post.
My needs are the same as many, I need to make a Mouse Cursor to control the entire OS. My research lab has develop a system that translate Tongue movements into commands, and now we want to make it compatible with Android. Following your example CursorOverlay app I was able to make a perfect translation from the Tongue Input Device to the cursor, to move the cursor anywhere on the screen(THANK!). However I am stuck at the clicking (inject touch event).
I applied Method 1 & 2 and confirmed they cursor works inside the Service App, however I get the security INJECT_EVENT error as you stated. I followed the recommendations of others to install the app at the system level but that did not help, since its a signature issue not a system permission. So my only option is Method 3, which I don’t understand with my lack of Linux background. So is it possible for you to guide me on how to achieve this. How to determine which event# is the current one for me or how to use your NativeInput class to make it work on my Service app.
This app will lead to a great innovation for people that really need this. Feel free to contact me at my email.
Phone: Galaxy S2 (SGH-T989)
OS: 4.0.3
January 10th, 2013 at 9:57 am
That’s a new way of controlling a mobile device with one’s tongue. I’ve seen your devices mounted inside one’s mouth – impressive idea. I will get back to you by email for your questions.
January 13th, 2013 at 10:26 pm
Hey Radu
Thanks for the quick reply.
I reply to the email you send me. I’ll be awaiting your reply.
Thanks.
January 24th, 2013 at 5:55 am
[...] wouldn’t work in JellyBean as it appears they removed touch event injection from the API. This guy has a good detailed blog post about different methods for injecting events, but ultimately [...]
January 26th, 2013 at 8:25 pm
Hi Radu, thank you for both your Android Cursor Overlay and Android Injecting Events Programmatically tutorials. I was really helped by your tutorials to finish my thesis. But I encounter a problem similar to Abner’s. I would like to create a mouse cursor that can be controlled by user and imitate the touch event.
Unfortunately, I don’t know how to do it with your 3rd method. I successfully tried the first and second method. But the third one only produces key event, not touch. Could you help me to give instructions to inject touch event with the 3rd method?
Because I think it’s the only way to inject touch event outside my application. I’ve tried the 1st and 2nd event with system permission (INJECT_EVENTS permission and install in /system/app/) and Android platform key signing, but it doesn’t work. It won’t install because the key is different from the platform I used.
My device is Samsung Galaxy S2 (GT-I9100) Rooted with SpeedMod ICS 4.0.4. I really appreciate your help. Thank you.
January 26th, 2013 at 10:01 pm
@Mike, I’ve been working on a library to implement method #3 including touch events, and input event node detection, as an attempt to help Abner, but also others interested on this perfectly normal functionality that google excluded on purpose.
I didn’t have time to structure a new article yet , but the library is available under GPL v2 here: http://code.google.com/p/android-event-injector/
For rooted phones, this is a good way to go. The library also includes the chmod part for the input event nodes.
January 27th, 2013 at 7:37 pm
Hi Radu, thank you for your quick reply
I’ll look into the library for the solution. Really appreciate your effort. Thank you, thank you very much.
January 28th, 2013 at 12:39 am
No problem, Michael, I try to help where possible. Just keep in mind that I still need to polish the library and also write an article on it, or at least a short wiki page.
January 28th, 2013 at 6:49 pm
Hi Radu, thanks for the library
Unfortunately I encounter an error when I build the native library (libEventInjector.so) with Android NDK using this command /ndk-build. I have errors showing that many of the constants, such as INPUT_PROP_POINTER, INPUT_PROP_DIRECT, INPUT_PROP_BUTTONPAD, etc., are undeclared. Thus, I can’t create the .so library file. Could you help me with this problem? Thank you.
January 28th, 2013 at 6:52 pm
Yes. Did you change the OS version of my original project? If not, make sure you are using the latest NDK version (android-ndk-r8b). The errors you are getting are caused by the compiler not being able to find resources related to the project platform, in this case the linux\input.h file.
January 28th, 2013 at 6:55 pm
If you would like to see the errors, you can see it here http://pastebin.com/z189Zkp3
January 28th, 2013 at 6:58 pm
I used Mac OS X for development. But I don’t think I change the OS version of your original project. The original project seems to be using Windows, correct? I’m using the latest NDK, which is android-ndk-r8c. I see. How do I include that input.h to my project?
January 28th, 2013 at 7:00 pm
First, browse to android-ndk-r8b\platforms\ and tell me what are the platforms listed there. Next search for input.h and let me know if all the platforms contain it.
January 28th, 2013 at 7:08 pm
There are platforms of 3, 4 5, 8, 9, 14 and all of them have linux/input.h file.
January 28th, 2013 at 7:17 pm
ok, this is good, now go to your SDK folder and make sure you have android-8 installed: Android\android-sdk\platforms\
January 28th, 2013 at 7:21 pm
Ah, there it is! I don’t have android-8. I’ll install it right away. Is it possible to use another platform, though?
January 28th, 2013 at 7:57 pm
I’ve downloaded it and repeated the process, but the errors still occur.
January 28th, 2013 at 8:00 pm
See that you can open the JAVA part of the project in eclipse and compile. I have the impression something went wrong with your Android-8 installation.
Yes, you can change the platform, and this is another way to go, simply edit AndroidManifest.xml and project.properties.
January 28th, 2013 at 9:55 pm
Hi Radu, thank you for your help
I managed to overcome the problem. Here are the steps:
1. Replace the input.h in the linux folder of Android NDK platform that you use, in my case it’s the android-14in android-ndk-r8c, with the more complete version of input.h file from https://github.com/quarck/csetup/blob/master/csetup/jni/input.h. The path is /platforms/android-14/arch-arm/usr/include/linux/input.h. It will clear all the errors of undeclared constants
2. Modify the methods’ names in the native C files with your own customized project package name. In this case, it’s the “Java_net_pocketmagic_android_eventinjector_Events_METHODNAME” because the file is located in package net.pocketmagic.android.eventinjector and in Events file with respective method’s name.
3. Make sure that the project target in project.properties file is showing one of SDK versions available in the NDK with the replaced input.h file. In my case, it’s the android-14. Just ignore the ndk-build command’s warning saying something like APP_PLATFORM is higher than the minSDK in manifest file.
4. Run the ndk-build program in your project’s root folder. Then “obj” folder will be created containing the .so library file. It will be also available in the “libs” folder.
5. Run the application from Eclipse just like usual. Note that, every time you make change to the native file, you must re-build the native library.
That’s it. Your project should now work. Thank you Radu. Couldn’t do it without you.
January 28th, 2013 at 10:01 pm
Michael, I don’t understand why some of the steps were necessary in your case (like 1 and 2), however I’m glad to see you managed to use it.
January 30th, 2013 at 1:04 pm
Hi Radu, the first step is required because the input.h file from the NDK that I have is not somewhat complete. It doesn’t have many variables that you mentioned in the native .h file. I don’t know why either. I’ve tried with NDK 8, 8b, 8c and all produce same error result. Maybe you had modify it previously on your computer. I don’t think it’s because the OS (I us Mac), but it probably is.
The second step is required because I copy and use it in my new project. If you only want to try to run the sample library project, you don’t need to do that
Anyway, thanks Radu.
But there is something that I want to ask. When I “open” the touchscreen input device from your project and try to inject touch event, my original touch screen is not working anymore. I can’t touch buttons or navigate through the Android OS. The only thing to recover it is by restarting the phone. Is that intentionally, expected, and okay?
January 30th, 2013 at 1:12 pm
@Michael, if you include this in your project, don’t forget that my code is under GPL license. Make sure you read it and understand it correctly. Normally you should include the code as it is, without changing names – this is not needed.
Modifying the NDK files is not ok. I compiled the code without changing anything , so probably there is something wrong at your end. Double check and triple check your NDK installation to make sure the paths are ok.
Regarding your question, I didn’t see this behavior, but I will double check when I get some time. You probably also need to close any open input event nodes after you are done with them to avoid any concurrency issues with the OS trying to access them. Just a random thought.
January 30th, 2013 at 1:36 pm
Hi Radu, yes I include it in my project and also put your name and website links as my sources and references. Okay, then. I need to adjust some methods to fit my project. I don’t change the name of the files, though.
Hmm, isn’t it necessary to modify the NDK files’ methods if I put your files in my package? Because the names of the packages are different, resulting different names of methods in NDK files. I’m sure the NDK installation is okay now because I can use ndk-build command in your sample project too.
I see, thank you Radu. I tried this in both your sample project and my project, and was still having same result. I couldn’t use my touch screen anymore as soon as I injected the touch event. Don’t know why. I’m trying to figure it out now.
January 30th, 2013 at 4:12 pm
[...] part 1 of this article I have indicated three ways of injecting events on the Android platform: Method 1: [...]
February 11th, 2013 at 7:46 am
I need to create an app that will automatically play a third party game.so the app should inject events to the game app. By third party games I mean I dont have the source code.So is it possible to inject events through an app to another app..?
February 25th, 2013 at 1:10 pm
I am a student of computer engineering.I am creating vnc server for android.I am working on a service(running in background) that performs touch events on given coordinates. I tried running the .apk provided by you at “https://code.google.com/p/android-event-injector/” this link, on a rooted emulator made at API level-10 as a system app ,but it crashed. Please help.
February 25th, 2013 at 1:40 pm
@shanu , you have full access to the source code, why don’t you check that error to see what exactly goes wrong?
February 25th, 2013 at 1:40 pm
@Shyam yes, that is possible. This is the purpose of this article.
February 25th, 2013 at 1:46 pm
i am using 3rd method on a rooted emulator at API level-10.but it is crashing. please help.
February 25th, 2013 at 3:19 pm
hi Radu,thankss for the quick reply !
could you suggest me the version of android at which your app runs most efficiently
February 25th, 2013 at 3:22 pm
Hi shanu, if I remember correctly, you can use android 2.2 or newer. But take a look at the source code posted on google-code to see the project’s target OS.
February 26th, 2013 at 10:21 am
hey Radu, finally the code is working on my rooted emulator. The code was crashing because there was no file as /dev/input/event3 in the emulator. There was only one file as /dev/input/event0. So, i changed the path in ur code and it started working !!!
,,, Really thankxx a lot !!
Also, i wanted it to perform touch on some given coordinate and i’ve successfully changed it for the same .
Thank u soo much
March 2nd, 2013 at 11:17 am
Sorry for a quick question, I did not test the code yet
It is about Method 1: Using internal APIs
You say it is able to send touch events inside our own application only
What about the soft keyboard that pops out?
Can we send touch events to the soft keyboard that pops out in our application without crashing?
Thanks in advance.
March 2nd, 2013 at 11:44 am
Sorry for a quick question, I did not test the code yet
It is about Method 1: Using internal APIs, You say it is able to send touch events inside our own application only
What about the soft keyboard that pops out? Let say Android stock keyboard, or Swype, or Swiftkey
Can we send touch events to the soft keyboard that pops out in our application without crashing?
I am not sure whether the soft keyboard is considered as the same application, or external application.
Actually I want to send touch events to Swype keyboard, swiping across several keys to create a word.
The inputs will come from my app, which will get the touch coordinate from front camera by detecting hand motion. (just a concept, not yet started working on it yet)
Thanks in advance.
April 5th, 2013 at 9:39 pm
Hi,
Your examples are very explanatory and work very well. Thank you very much!
I have a question about a problem that is puzzling me.
Is there any way to supress some input events so Android does not handle them?
The idea would be, for instance, polling a device for it’ s events and generate other transformed events injected in another device out of those (i.e. keyboard to touch). The original input device events should not be handled by android. To be honest I’m not sure if this can be done at a kernel level or if there is any way to access the configuration of android so it stops listening to events from a particular input.
May 9th, 2013 at 7:45 am
Hello, great post. I have a similar question as tcboy. I am trying to simulate touch gestures over the software keyboard to swipe across several keys to make a word. I am trying to use your method 2 and the below code from Robotium to interact with my app. It works fine when the keyboard in not up but when the software keyboard is up it is executed but no MotionEvent.ACTION_DOWN is registered in the app of the keyboard.
I was able to get adb shell input tap x y to click on the key in the software keyboard but this will not work as I need more control of the specific motion actions.
Any suggestion on how I can get suggestion 2 working on the software keyboard to press keys? Do I need to set some flag in my app layout.xml file?
TEST CODE:
Instrumentation inst = getInstrumentation();
// sending event – finger touched the screen
MotionEvent event = MotionEvent.obtain(downTime, eventTime, MotionEvent.ACTION_DOWN, xStart, yStart, 0);
inst.sendPointerSync(event);
May 9th, 2013 at 10:28 am
see part 2 of this article