5

I'm trying to write a function to allow me to send a left click to x, y co-ordinates on screen without my mouse cursor moving.

I've read through the send message documentation and the mouse input notification documentation and I've come up with a few different approaches, none of which work. And none throw an error.

I'm using win32gui.FindWindow to get the hwnd and then I've tried using both PostMessage and SendMessage to perform the click, none so far work.

import win32api, win32gui, win32con

def control_click(x, y):

    hWnd = win32gui.FindWindow(None, "Pixel Starships")
    l_param = win32api.MAKELONG(x, y)

    if button == 'left':
        win32gui.PostMessage(hWnd, win32con.WM_LBUTTONDOWN, win32con.MK_LBUTTON, l_param)
        win32gui.PostMessage(hWnd, win32con.WM_LBUTTONUP, win32con.MK_LBUTTON, l_param)
   
control_click(526, 694)

def click(x, y):
    hWnd = win32gui.FindWindow(None, "Pixel Starships")
    lParam = win32api.MAKELONG(x, y)

    win32api.SendMessage(hWnd, win32con.WM_LBUTTONDOWN, win32con.MK_LBUTTON, lParam)
    win32api.SendMessage(hWnd, win32con.WM_LBUTTONUP, win32con.MK_LBUTTON, lParam)

click(526,694)

def leftClick(pos):
    hWnd = win32gui.FindWindow(None, "Pixel Starships")
    lParam = win32api.MAKELONG(pos[0], pos[1])

    win32gui.SendMessage(hWnd, win32con.WM_LBUTTONDOWN, win32con.MK_LBUTTON, lParam) 
    win32gui.SendMessage(hWnd, win32con.WM_LBUTTONUP, 0, lParam)

leftClick([526,964])

What do I need to do to get this function to work?

mdurant
  • 27,272
  • 5
  • 45
  • 74
mak47
  • 303
  • 2
  • 14
  • Have you tried to add a thread which periodicaly retrieves that position? – Dominik Lovetinsky May 09 '22 at 14:34
  • I'm not sure why I would want to do that, maybe my question wasn't clear sorry! I'm trying to write a function that will allow me to click without moving the mouse. The specific example above is just an example, the co-ordinates will change depending on the requirement. – mak47 May 09 '22 at 14:37
  • Checkout this https://stackoverflow.com/questions/12363215/send-mouse-click-message – Haseeb Mir May 09 '22 at 14:40
  • That looks like the exact approach I've outlined in my question that doesn't work @HaseeBMir unless I'm doing something wrong, which I don't think I am, at least based on that post. – mak47 May 09 '22 at 14:46
  • 1
    @mak47 i have done that in Python using import Windows api methods from DLL, you can check in C++ how its done : https://github.com/haseeb-heaven/GTLibc/blob/master/GTLibc.c#L3597 and then i have imported this c++ into python and its working – Haseeb Mir May 09 '22 at 14:54
  • 1
    You have to use [Send-Input](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-sendinput) method instead of SendMessage or PostMessage those are for Win32 Dispatch Loop. – Haseeb Mir May 09 '22 at 15:01
  • I'll try send-input, can't really follow the github link! Bit too advanced for me maybe! – mak47 May 09 '22 at 15:29
  • After a few days of research, @HaseeBMir Send-Input moves the mouse. I need the mouse not to move. – mak47 May 13 '22 at 06:20
  • @mak47 you can use [AHK](https://www.autohotkey.com/) script like this **SendEvent {Click x,y}** without moving mouse. but if you really need python then have to figure out something else – Haseeb Mir May 13 '22 at 14:09
  • Just refactored a few thousands lines of AHK into python (learning as I go!) so I can use OpenCV rather than AHK tools like FindText, rather not have to go back to AHK! – mak47 May 13 '22 at 14:35
  • imo AHK is far better choice if you want automation it has also read Pixel screen technology and can click on basis on pixels if you want, i would recommend to use AHK instead of OpenCV. – Haseeb Mir May 13 '22 at 21:18
  • you can use inspect.exe(https://learn.microsoft.com/en-us/windows/win32/winauto/inspect-objects#:~:text=Inspect%20(Inspect.exe)%20is,Active%20Accessibility%20(MSAA)%20properties.) to checkout the ui control, if it has 'invokepattern', then you can invoke the pattern on that control , don't need move mouse, use UIA API – justin Jul 26 '22 at 10:10
  • my solution is run the desired program in windows sandbox, and use the virtualized mouse. – Valen Nov 14 '22 at 16:59

4 Answers4

1

I have been looking for an answer to this question for over 2 years. I have tried all possible scripts, nothing worked (well, pywinauto works fine in a lot of cases, but with some apps, it doesn't). I found something yesterday that I want to share with you. It is not a script for mouse clicks, but for keystrokes. However, it might be the way to solve your (our) problem. I have tested it already with some apps that gave me trouble sending keystrokes, and it worked like a charm. I hope I can get it to run with mouse clicks. Keep us informed if you find something, I will also keep looking for a solution.

https://github.com/byt3bl33d3r/Invoke-AutoIt

Loads the AutoIt DLL and PowerShell assemblies into memory and executes the specified keystrokes.

This was mainly created to bypass some application restrictions when it came to sending keystroke input via sendkeys(), sendinput() or sendmessage(). e.g. Windows's Remote Desktop Client ;)

Edit: I found the solution! It is possible, but you need to know assembly! Have a look at this video that I have just found https://www.youtube.com/watch?v=T5sXoEEPFBQ

For the ones that don't know assembly, and need to use one of the common tools, here is my experience:

Pywinauto: Works best for sending "clicks" to a background window, but many times it doesn't work like you expected. https://pywinauto.readthedocs.io/en/latest/

AHK: The most reliable mouse clicker is, in my opinion, AHK: https://pypi.org/project/ahk/ (This is, for example, the only mouse clicker that I found which works with Roblox). However, it doesn't work well with background windows (at least for me)

Hans
  • 148
  • 2
  • 7
  • 1
    I feel like we're both trying to basically figure out the same sort of thing taking a look at both our questions! Let me know if there's a way we can connect, discord or whatever I'm sure we might be able to help each other out! – mak47 Aug 08 '22 at 14:40
  • Of course! I would love to! aulasparticularesdealemaosp@gmail.com I have been studying assembly for last 2 weeks! hahaha – Hans Aug 10 '22 at 23:41
-1

Could this work?

win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN,x,y,0,0)
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP,x,y,0,0)

I would also recommend trying pyautogui. It might help.

Seth Edwards
  • 47
  • 1
  • 7
  • Big fan of pyautogui but the mouse move functions all require moving the actual mouse, you can feed in x,y co-ordinates but the mouse pointer snaps to the location to perform the `click`. I use it extensively and I'm trying to get around the mouse pointer ever moving. Will try the win32api.mouse_event now. – mak47 May 17 '22 at 10:24
  • No click unforunately – mak47 May 17 '22 at 11:01
  • What version of Windows are you using? Version,Build,64/32 bit? – Seth Edwards May 17 '22 at 19:21
  • Windows10, 64, most recent build, everything is up to date and nothing outstanding. – mak47 May 18 '22 at 07:15
-1

I would also recommend using pyautogui because afaik it has a function to generate a click at a specific position without changing current cursor location.

Also you probably want to have a couple of milliseconds between button-down and button-up function because windows sometimes tend to not register the second button function due to beeing to fast after the button down. Additionally if this is for a game (pixel starships), you want to randomize this milliseconds waitingframe. Otherwise you might run into anti-bot checks and get dispelled/banned from the game server if pixel starships implements those.

4lexKidd
  • 158
  • 1
  • 1
  • 9
  • 1
    as a side note: pyautogui also allows for jpg search in a given frame. This means if you search for click locations on an element that always looks the same, you can just search for it on the screen and use the x, y coordinates of the found instances of the searched-for jpg. For example if you want to click on specific skill icons or enemy ships of specific look. This function also allows for "closest sibling" style search if your searched element has slight changes or might be overlapped by some other elements. – 4lexKidd May 16 '22 at 21:19
  • Big fan of pyautogui but the mouse move functions all require moving the actual mouse, you can feed in x,y co-ordinates but the mouse pointer snaps to the location to perform the `click`. I use it extensively and I'm trying to get around the mouse pointer ever moving. – mak47 May 17 '22 at 10:24
  • before the click you can easily get your current mouse position and after the click, you can snap back the current position – 4lexKidd May 17 '22 at 11:53
  • Yes, but I don't want to do that. My requirement is explicitly with no movement from the mouse whatsoever, perform a click. I don't want to move and snap back, I don't want to any movement whatsoever. – mak47 May 17 '22 at 11:56
  • hmm that might be tough to achieve. Afaik the in-build click event is tied to the mouse position and de-coupling it would require you to build your own event handler that throws the correct "onClicked" style event in the ui elements. My advice is: have a look at pyautoguis click function to possibly replicate a un-coupled version. – 4lexKidd May 17 '22 at 12:02
  • Oddly there are a lot of answers to this question out there but none of them actually work which I find really weird. I've crawled through all the relevant stackposts and none of them work and none of them throw errors and none and yet all the answers make it sound simple. – mak47 May 17 '22 at 12:05
  • Also apparently the inbuilt click isn't tied to mouse position, they're two separate elements. – mak47 May 17 '22 at 12:06
  • oh well my bad, i don't know pyautoguys internal lib code that well. But in this case just call the generic-click function with propper coorinates and it should generate a click event at that location. If the cursor still moves there it is,as i mentioned, tied to the cursor positions and would need a custom decoupled function call. – 4lexKidd May 17 '22 at 12:12
-3

Maybe you can try with Pywinauto is another library similar to PyAutoGUI, but allows you to manipulate apps via her control identifier.

import pywinauto
app = pywinauto.Application().connect(path='notepad.exe')
app.UntitledNotepad.MenuSelect("Edit->Replace")
Alberto Diaz
  • 1
  • 3
  • That function allows for interaction with application elements rather than sending clicks. The application in question doesn't have the facility, unfortunately. – mak47 May 19 '22 at 06:11