SikuliX automates anything you see on the screen of your desktop computer running Windows, Mac or some Linux/Unix. It uses image recognition powered by OpenCV to identify GUI components. This is handy in cases when there is no easy access to a GUI's internals or the source code of the application or web page you want to act on.
SikuliX supports as scripting languages
Python language level 2.7 (supported by Jython)
running RobotFramework text-scripts is supported (see docs)
Ruby language level 1.9 and 2.0 (supported by JRuby)
JavaScript (supported by the Java Scripting Engine)
… and you can use it in Java programming and programming/scripting with any Java aware programming/scripting language (Jython, JRuby, Scala, Clojure, …).
Though SikuliX is currently not available on any mobile device, it can be used with the respective emulators on a desktop computer or based on VNC solutions. A solution for Android devices based on ADB (Android Debugging Bridge) is in an early experimental stage,
Besides locating images on a screen SikuliX can run the mouse and the keyboard to interact with the identified GUI elements. This is available for multi monitor environments and even for remote systems with some restrictions.
SikuliX comes with basic text recognition (OCR) and can be used to search text in images. This feature is powered by Tesseract.
SikuliX is a Java application, that works on Windows, macOS and most Linux/Unix systems.
Germany
EU
Comments and Reviews
great app to automating. USP is the visual automation aspect (define buttons or others visual elements to click on). Thank you developers for this great app!