Android Overlay Mouse Cursor

By Radu Motisan Posted on July 27th, 2012 , 4468 Views (Rate 15.09)

For an application developer it is important to have total freedom in shaping the result of his work.
Even if Android OS has been designed without the mouse cursor in mind, there are plenty of situations where a developer needs a cursor on top of other applications installed.

We can draw the cursor at the framebuffer level, and while this provides optimum performance, it requires special privileges that regular apps don't have.
The second approach , and the subject of this article, is drawing a cursor using an Overlay.

What is an overlay?
The SDK defines them as: "Window type: system overlay windows, which need to be displayed on top of everything else. These windows must not take input focus, or they will interfere with the keyguard. "

So what we are about to do is have a small view created, make it show a cursor image, and move it around.
To make things easier, I created a special service class, since the code that displays a cursor would probably be a service , running in the background, and accepting cursor position updates.

Sample application
You can download the sample application here. You can use the code in free or non commercial applications.

Details
Unless your AndroidManifest.xml contains the SYSTEM_ALERT_WINDOW permission, the code will fail:

 
 WindowManager.LayoutParams params = new WindowManager.LayoutParams(
    WindowManager.LayoutParams.WRAP_CONTENT,
    WindowManager.LayoutParams.WRAP_CONTENT,
    WindowManager.LayoutParams.TYPE_SYSTEM_OVERLAY,
    WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE | 
WindowManager.LayoutParams.FLAG_NOT_TOUCHABLE |
WindowManager.LayoutParams.FLAG_LAYOUT_IN_SCREEN, 
PixelFormat.TRANSLUCENT);

To make the cursor be able to cover the status bar, and have it's coordinates starting from screen's (0,0) leftmost and topmost point, you will need to specify the .FLAG_LAYOUT_IN_SCREEN flag.

Hope this code proves useful. Looking forward to your comments, questions and feedback,

Customizing the system's pointer

It is possible to detect the system mouse coordinates and draw a custom overlay on top of the system mouse cursor. This way we can increase the pointer image size, a very useful enhancement for those that have problems seeing the tiny default cursor. Here is a demo showing a bluetooth mouse cursor and our bigger custom mouse cursor overlay.

Mike:
September 14th, 2012 at 1:59 pm

Default HVGA? Can it change resolution?

Radu Motisan:
September 14th, 2012 at 3:41 pm

Mike:
September 15th, 2012 at 12:21 am

set params.width in service’s onCreate()

Radu Motisan:
September 15th, 2012 at 9:04 am

change it to fit your needs

Jesper:
October 21st, 2012 at 9:51 pm

Thanks for a great article!

Radu Motisan:
October 21st, 2012 at 11:30 pm

I’m glad this helps.

Surbhi Jain:
October 31st, 2012 at 1:34 pm

Great Article ! thanks

Radu Motisan:
October 31st, 2012 at 1:36 pm

@Surbhi: you’re welcome. what are you working on?

Thomas:
November 25th, 2012 at 8:07 pm

Hello,

is it possible to ensure that the cursor is positioned at the location where the user touches the screen?

Thank you in advance

Radu Motisan:
November 26th, 2012 at 8:08 am

@Thomas, yes, it is possible, feel free to contact me for more details.

Rafael Orágio:
December 26th, 2012 at 11:58 pm

Is it possible generate a “tap screen” using this cursor?

Thanks!

Radu Motisan:
December 28th, 2012 at 11:45 am

@Rafy yes, it is possible, see: http://www.pocketmagic.net/2012/04/injecting-events-programatically-on-android

Michael James:
January 1st, 2013 at 5:16 pm

Just what I need! Thank you for this sample application. It really helps me a lot. I appreciate it so much. You don’t know how much it means to me Once again, thanks.
FYI, I’m creating application to control the mouse with camera for my thesis. Wish me luck

Radu Motisan:
January 2nd, 2013 at 12:04 pm

@Michael: Good luck with your thesis. Would love to see your project . Did you have the chance to see the “HeadMouse” project? It is still available as open source.

Michael James:
January 30th, 2013 at 12:53 pm

@Radu: Thank you Radu. I’ll let you see as soon as it finishes Haven’t seen the project. Is it yours? Where can I see it? Thanks.

Programmatically Injecting Events on Android – Part 2 « PocketMagic:
January 30th, 2013 at 4:14 pm

[...] custom mouse pointer on top of your Android screen, for application control purposes, as presented here, that also needs to be able to inject the touch events when the user clicks. Not to mention various [...]

Graham:
February 26th, 2013 at 3:05 pm

Is it possible to get the standard mouse cursor arrow position as in framework and using the overlaying to drawn new cursor over it. I cannot fine any means of getting the mouse coordinates (x,y)

Radu Motisan:
February 26th, 2013 at 6:30 pm

@Graham, yes it is possible. I already did this for a client. Drop me a mail for more details.

Graham:
February 27th, 2013 at 11:32 am

Thanks Radu, We are trying to resolve a problem for disabled users to give them a bigger, brighter mouse arrow instead of the stock one which is difficult to see. I would appreciate more details as you have offered. Can you send to my email address, thanks.

Ragunath Jawahar:
February 28th, 2013 at 4:44 pm

Hi Radu, I have a question. How do I capture events from an IR remote and deliver them to the overlay?

Daniel Leal:
March 7th, 2013 at 7:04 pm

Hello Radu,

Thanks for showing your work, it’s really great!

I’m trying to use it to have a remote touchpad app between android devices, and I have a problem. The cursor does not cover the whole screen, it dissapears on the right hand side. What’s strange is that it cover the complete height of the screen, the problem it’s just width-wise. Do you know why this could be happening? I have tried your project from the scratch and also have tried adapting it for my code; same result in both cases.

Thank you very much in advance!

Radu Motisan:
March 7th, 2013 at 7:36 pm

@Ragunath: are you working for Math R.?

@Daniel: in my code find:
…
WindowManager.LayoutParams params = new WindowManager.LayoutParams(
WindowManager.LayoutParams.WRAP_CONTENT,
WindowManager.LayoutParams.WRAP_CONTENT,
WindowManager.LayoutParams.TYPE_SYSTEM_OVERLAY,
…

Change WRAP_CONTENT to MATCH_PARENT, or the number of pixels representing your Screen_width. How do you plan to inject the clicks?

Daniel Leal:
March 8th, 2013 at 11:01 am

@Radu: Thank you very much, it works! For the click injection I’m using your second method from http://www.pocketmagic.net/2012/04/injecting-events-programatically-on-android/ . I am working with a rooted device, so I can install the app with system signature, as a system app, and use the INJECT_EVENTS permission.

By the way, the cursor is not showing above the system bar, though I have the FLAG_LAYOUT_IN_SCREEN flag. These are the params I’m using:

WindowManager.LayoutParams params = new WindowManager.LayoutParams(
WindowManager.LayoutParams.MATCH_PARENT,
WindowManager.LayoutParams.MATCH_PARENT,
WindowManager.LayoutParams.TYPE_SYSTEM_ALERT,//TYPE_SYSTEM_ALERT,//TYPE_SYSTEM_OVERLAY,
WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE | WindowManager.LayoutParams.FLAG_NOT_TOUCHABLE |
WindowManager.LayoutParams.FLAG_LAYOUT_IN_SCREEN //will cover status bar as well!!!
,PixelFormat.TRANSLUCENT);

Really nice work, keep it up!

Radu:
March 8th, 2013 at 12:29 pm

@Daniel: double check my code. As you seen in the video, I have no problem in getting the cursor above the system bar.

Daniel Leal:
March 8th, 2013 at 5:01 pm

@Radu: I’ve seen the video, but there’s no system bar in it. By system bar I’m refering to the bar that appears from ICS at the bottom of the screen, with the BACK, HOME and RECENT buttons. I can inject events there (click on those buttons), but no pointer is shown.

Radu:
March 8th, 2013 at 10:13 pm

@Daniel, are you referring to a tablet? Better drop me a screen capture showing this issue. I didn’t have such an issue, and I recall testing the custom pointer on lock screen (the pull ring screen), on the notifications screen, titlebar, etc. But it is true I don’t remember testing on a tablet.

Daniel Leal:
March 11th, 2013 at 3:02 pm

@Radu: sorry, for the delay, took the weekend off, hehehe. Yes, I’m referring to a tablet. Should we communicate through email better? To avoid filling the comments section. It would be easier to send you the image too.

Oh, by the way, I was trying to inject multitouch events, to simulate a pinch for instance, but although the events are injected (shown in developer mode), no pinch gesture is recognised by, for instance, Google Maps. Do you know why this could happen?

Daniel Leal:
March 12th, 2013 at 5:21 pm

I succeeded with the multitouch injection. The key was to set the ‘pressure’ field from the PointCoords to 1. It seems that by default it is considered as 0, so no “touch” is computed after all.

Still working on the ‘cursor over system bar’ mission

Ovidiu:
March 20th, 2013 at 7:48 pm

Buna Radu, mersi de solutie, as avea o intrebare, incercam sa trimitem un event de click din overlay catre android prin partea de software, ai ideie cum am putea asta fara a avea permisiune de root la sistem?

Mersi mult

Radu Motisan:
March 20th, 2013 at 7:50 pm

Salut Ovidiu,

Citeste http://www.pocketmagic.net/2012/04/injecting-events-programatically-on-android/ si http://www.pocketmagic.net/2013/01/programmatically-injecting-events-on-android-part-2/

Fara root nu se poate. Ar putea exista alte alternative, dar depinde de ceea ce incerci sa faci. Asa ca poate vrei sa dai mai multe detalii.

Radu

Ovidiu:
March 21st, 2013 at 8:22 am

Mersi de raspuns Radu! Ce am vrea sa facem e exact ce ai expus aici, o aplicatie software care sa simuleze un mouse pe ecran fara a avea conectat un mouse la exterior, mouse-ul fiind simulat pe ecran de aplicatie si controlat doar prin touch screen

Radu Motisan:
March 21st, 2013 at 10:34 am

Poti folosi codul acesta sau cel pentru injectare de evenimente, gratuit, dar numai in aplicatii necomerciale.
Din pacate la partea de injectare ai nevoie de root.

Succes

Aruna:
March 28th, 2013 at 1:29 pm

I need to get the TYPE_SYSTEM_ALERT or similar overlay type that stay on top of all the other window and also gets the motion events. At the same time passes the motion events to any overlay under the top layer. Is it possible to do this? If so can you please point me to a good documentation or show me how to do this. Thanks in advance.

Radu Motisan:
March 28th, 2013 at 1:30 pm

@Aruna yes it is possible, see the video at the bottom of this article.

Aruna:
March 28th, 2013 at 1:43 pm

Thanks for the reply. Is this just by getting the correct LayoutParams combination or does it need event injections as well. I’ve tried different LayoutParams combinations but failed to pass the events to the layers under the top layer as the top layer getting the focus.

Radu Motisan:
March 28th, 2013 at 1:48 pm

@Aruna I sent you an email.

Shawn:
May 8th, 2013 at 8:32 pm

Hey Mr.Radu. The sample codes you provided seems have some problem with resolution. I assume the overlay width and height is define with params but change it does not affect the cursor reachable area. What I am doing is connect two device (with different resolution) and one device send the co-ordinates to update the position of cursor. However I am not sure why the cursor area is automatically set as client and limit the cursor reachable area (480*800, use to send co-ordinate. and server is 720 * 1280). Do you know what cause this happen and know the possible solutions?
Sincerely,Shawn

Shawn:
May 8th, 2013 at 8:37 pm

And I try connected two devices with (480*800 and 480*840 resolutions) the cursor reachable area at 480*840′s device is 480*800 also

Shawn:
May 8th, 2013 at 8:43 pm

I am sorry….it’s my own mistake…..I forgot I have set the update border to avoid cursor update to outside screen and forgot verfiy the algorthm. looks this is the problem. I’m Sorry for disturb.

wangpeng1:
May 11th, 2013 at 2:45 pm

can you share the mouse control the cursor overlay?

Mouse control the mobile Cursor overlay - How-To Video:
May 11th, 2013 at 2:54 pm

[...] . but the author only make overlay on the mobile screen. the link is http://www.pocketmagic.net/2012/07/an…; . i want to know how to finished it,can you give some advice. if i need some c code of android [...]

PocketMagic

Top Articles! See PocketMagic on Facebook