Full Code of Nekose/Mouseomate for AI

Repository: Nekose/Mouseomate
Branch: master
Commit: a35b0dc95e80
Files: 9
Total size: 17.8 KB

Directory structure:
gitextract_z12f3spn/

├── .gitignore
├── LICENSE
├── README.md
├── images/
│   └── __init__.py
├── main.py
├── requirements.txt
└── src/
    ├── image_handler.py
    ├── mouse_automate_black.py
    └── mouse_automate_color.py

================================================
FILE CONTENTS
================================================

================================================
FILE: .gitignore
================================================
/src/__init__.py
*.pyc


================================================
FILE: LICENSE
================================================
Creative Commons Legal Code

CC0 1.0 Universal

	CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE
	LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN
	ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS
	INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES
	REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS
	PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM
	THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED
	HEREUNDER.

Statement of Purpose

The laws of most jurisdictions throughout the world automatically confer
exclusive Copyright and Related Rights (defined below) upon the creator
and subsequent owner(s) (each and all, an "owner") of an original work of
authorship and/or a database (each, a "Work").

Certain owners wish to permanently relinquish those rights to a Work for
the purpose of contributing to a commons of creative, cultural and
scientific works ("Commons") that the public can reliably and without fear
of later claims of infringement build upon, modify, incorporate in other
works, reuse and redistribute as freely as possible in any form whatsoever
and for any purposes, including without limitation commercial purposes.
These owners may contribute to the Commons to promote the ideal of a free
culture and the further production of creative, cultural and scientific
works, or to gain reputation or greater distribution for their Work in
part through the use and efforts of others.

For these and/or other purposes and motivations, and without any
expectation of additional consideration or compensation, the person
associating CC0 with a Work (the "Affirmer"), to the extent that he or she
is an owner of Copyright and Related Rights in the Work, voluntarily
elects to apply CC0 to the Work and publicly distribute the Work under its
terms, with knowledge of his or her Copyright and Related Rights in the
Work and the meaning and intended legal effect of CC0 on those rights.

1. Copyright and Related Rights. A Work made available under CC0 may be
protected by copyright and related or neighboring rights ("Copyright and
Related Rights"). Copyright and Related Rights include, but are not
limited to, the following:

  i. the right to reproduce, adapt, distribute, perform, display,
	 communicate, and translate a Work;
 ii. moral rights retained by the original author(s) and/or performer(s);
iii. publicity and privacy rights pertaining to a person's image or
	 likeness depicted in a Work;
 iv. rights protecting against unfair competition in regards to a Work,
	 subject to the limitations in paragraph 4(a), below;
  v. rights protecting the extraction, dissemination, use and reuse of data
	 in a Work;
 vi. database rights (such as those arising under Directive 96/9/EC of the
	 European Parliament and of the Council of 11 March 1996 on the legal
	 protection of databases, and under any national implementation
	 thereof, including any amended or successor version of such
	 directive); and
vii. other similar, equivalent or corresponding rights throughout the
	 world based on applicable law or treaty, and any national
	 implementations thereof.

2. Waiver. To the greatest extent permitted by, but not in contravention
of, applicable law, Affirmer hereby overtly, fully, permanently,
irrevocably and unconditionally waives, abandons, and surrenders all of
Affirmer's Copyright and Related Rights and associated claims and causes
of action, whether now known or unknown (including existing as well as
future claims and causes of action), in the Work (i) in all territories
worldwide, (ii) for the maximum duration provided by applicable law or
treaty (including future time extensions), (iii) in any current or future
medium and for any number of copies, and (iv) for any purpose whatsoever,
including without limitation commercial, advertising or promotional
purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each
member of the public at large and to the detriment of Affirmer's heirs and
successors, fully intending that such Waiver shall not be subject to
revocation, rescission, cancellation, termination, or any other legal or
equitable action to disrupt the quiet enjoyment of the Work by the public
as contemplated by Affirmer's express Statement of Purpose.

3. Public License Fallback. Should any part of the Waiver for any reason
be judged legally invalid or ineffective under applicable law, then the
Waiver shall be preserved to the maximum extent permitted taking into
account Affirmer's express Statement of Purpose. In addition, to the
extent the Waiver is so judged Affirmer hereby grants to each affected
person a royalty-free, non transferable, non sublicensable, non exclusive,
irrevocable and unconditional license to exercise Affirmer's Copyright and
Related Rights in the Work (i) in all territories worldwide, (ii) for the
maximum duration provided by applicable law or treaty (including future
time extensions), (iii) in any current or future medium and for any number
of copies, and (iv) for any purpose whatsoever, including without
limitation commercial, advertising or promotional purposes (the
"License"). The License shall be deemed effective as of the date CC0 was
applied by Affirmer to the Work. Should any part of the License for any
reason be judged legally invalid or ineffective under applicable law, such
partial invalidity or ineffectiveness shall not invalidate the remainder
of the License, and in such case Affirmer hereby affirms that he or she
will not (i) exercise any of his or her remaining Copyright and Related
Rights in the Work or (ii) assert any associated claims and causes of
action with respect to the Work, in either case contrary to Affirmer's
express Statement of Purpose.

4. Limitations and Disclaimers.

 a. No trademark or patent rights held by Affirmer are waived, abandoned,
	surrendered, licensed or otherwise affected by this document.
 b. Affirmer offers the Work as-is and makes no representations or
	warranties of any kind concerning the Work, express, implied,
	statutory or otherwise, including without limitation warranties of
	title, merchantability, fitness for a particular purpose, non
	infringement, or the absence of latent or other defects, accuracy, or
	the present or absence of errors, whether or not discoverable, all to
	the greatest extent permissible under applicable law.
 c. Affirmer disclaims responsibility for clearing rights of other persons
	that may apply to the Work or any use thereof, including without
	limitation any person's Copyright and Related Rights in the Work.
	Further, Affirmer disclaims responsibility for obtaining any necessary
	consents, permissions or other rights required for any use of the
	Work.
 d. Affirmer understands and acknowledges that Creative Commons is not a
	party to this document and has no duty or obligation with respect to
	this CC0 or use of the Work.


================================================
FILE: README.md
================================================
Mouseomate V0.8, Now with Color!

Original Code by Nekose

Color Implementation and GUI by Quadrum1

formating contributions by Logistic-bot


To use, run main.py. The image you would like to draw must be inside the 'images' folder. 

To switch between color or BW, change the mode variable near the top of main. If using color, specify the program you would like to draw with to specify color controls. (currently only supports MS Paint)

If running in black and white, images should ideally be vector-like, with limited shading or colorpallet (Although the program will attempt to monochrome any image given).

***
Example Of Use:

* After running, a prompt will ask which file to open. Once open a dialog box will appear asking for the following:

What is the approximate pixel size you would like to output?> 500
What is the approximate brush size in pixels? (Use 1 for one to one drawing)>? 1

* Once the parameters have been provided, the following message will be displayed:

Preview image loaded. Enter to begin 3 second countdown to start, N to abort. Once image has started, pulling mouse to the upper left corner will abort the program.

* Hitting enter will start a 3 second countdown, afterwhich the program will take control of the mouse, using the current position as the upper left corner of the image.

***

Known issues:
* Output speed appears to be limited by the windows input buffer. You may need to wait till the program finishes running before the image apppears. If you have CPU speed issues, you can try increasing "lsleep" and "rsleep" values at the top of main.py.

To Do:
* investigate better drawing algorithms (Line tracing with cv2.HoughLinesP()?)


================================================
FILE: images/__init__.py
================================================


================================================
FILE: main.py
================================================
import time
import sys
import os
from easygui import multenterbox

cwd = os.getcwd()
sys.path.append(cwd+'/src')

from image_handler import Image_handler
from mouse_automate_black import Mouseomate as Mouseomate_black
from mouse_automate_color import Mouseomate as Mouseomate_color

mode = "black" # "color", "black"
app = "paint" # What application are you drawing with?

os.chdir("images")


lsleep = 0.002 #adujust this to change pause time at end of line (recommend .002)
rsleep = 0.025 #adujust this to change pause time at end of row (recommend .025)
imagename = Image_handler.get_image()
handler = Image_handler(imagename)
message = "Resizevalue: What is the approximate pixel size you would like to output?\nOffset: What is the approximate brush size in pixels? (1 for one to one drawing)"
resizevalue, offset = multenterbox(message,"Mouseomate", ["Resizevalue","Offset"])
resizevalue = int(resizevalue) if resizevalue else 100
offset = int(offset) if offset else 1
resizevalue = resizevalue / offset
handler.convert_bandw(mode)
handler.resize(resizevalue)
	
handler.im.show()
returnkey = None
while returnkey == None:
	print("Preview image loaded. Enter to begin 3 second countdown to start, N to abort. Once image has started, pulling mouse to any corner will abort the program.")
	returnkey = input()
	returnkey = returnkey.lower()
	if returnkey == 'n':
		exit()
	if returnkey == 'i':
		handler.invert(imagename)
		handler.convert_bandw(mode)
		resizevalue = resizevalue / offset
		handler.resize(resizevalue)
		handler.im.show()
		returnkey = None

time.sleep(3)
array = handler.update_array()
if mode == "color":
	Mouseomate_color.image_to_lines(array, offset, rsleep, lsleep, app)
elif mode == "black":
	Mouseomate_black.image_to_lines(array, offset, rsleep, lsleep)
		
repeat = 'y'
while repeat == 'y':
	repeat = input("Y to redraw after 3 sec pause, or enter to exit")
	repeat = repeat.lower()
	if repeat == 'y':
		time.sleep(3)
		if mode == "color":
			Mouseomate_color.image_to_lines(array, offset, rsleep, lsleep, app)
		elif mode == "black":
			Mouseomate_black.image_to_lines(array, offset, rsleep, lsleep)
	else:
		exit()



================================================
FILE: requirements.txt
================================================
MouseInfo==0.1.3
numpy==1.22.0
pandas==1.0.3
Pillow==9.0.1
PyAutoGUI==0.9.50
PyGetWindow==0.0.8
PyMsgBox==1.0.7
pyperclip==1.8.0
PyRect==0.1.4
PyScreeze==0.1.26
python-dateutil==2.8.1
PyTweening==1.0.3
pytz==2020.1
six==1.14.0
easygui==0.98.1


================================================
FILE: src/image_handler.py
================================================
from PIL import Image, ImageFilter, ImageOps
import numpy as np
from easygui import fileopenbox

class Image_handler(object):
	def __init__(self,image):
		self.im = Image.open(image)

	@staticmethod
	def get_image() -> str:
		"""
		Ask's users the name of the file to be used
		:return:
		a string of the file name
		"""
		path = fileopenbox()
		return path

	def convert_bandw(self,mode) -> None:
		"""
		converts image to a black and white bit per pixel image
		:return:
		"""
		if mode == "color":
			self.im = self.im.convert(mode='RGB',dither=None)
		elif mode == "black":
			self.im = self.im.convert(mode='1',dither=None)

	def invert(self,image) -> None:
		"""
		Returns a color inverted image file
		:return:
		"""
		self.im = ImageOps.invert(Image.open(image))

	def resize(self,value:int) -> None:
		"""
		resizes image to fit withing an X by X square
		:param value:
		Size of the square input field
		:return:
		"""
		horizontalsize, verticalsize = self.im.size
		if horizontalsize > verticalsize:
			conversion = horizontalsize / value
			self.im = self.im.resize((int(horizontalsize / conversion), int(verticalsize / conversion)))
		else:
			conversion = verticalsize / value
			self.im = self.im.resize((int(horizontalsize / conversion), int(verticalsize / conversion)))

	def update_array(self) -> np.array:
		"""
		Converts image into a numpy array of bools
		:return:
		"""
		array = np.array(self.im)
		return array






================================================
FILE: src/mouse_automate_black.py
================================================
import numpy
import pyautogui
import time	  
class Mouseomate(object):
	pyautogui.PAUSE = 0.00 #useful
	pyautogui.FAILSAFE = True

	@staticmethod
	def image_to_lines(image_array: numpy.array, offset:int, rsleep:int, lsleep:int) -> None:
		"""
		Converts an image array to mouseclicks.
		:param image_array:
		A numpy array of bools, where False represents a click, and True represents no click.
		:param offset:
		An int which provides spacing between each pixel in image_array. Usefull to adjust for brush size used in whatever this will be outputting for.
		:param rlseep:
		int which designates how long in second the mouse will take drawing a line
		:param rsleep:
		int which designates how long in seconds to pause at end of row
		:return:
		"""
		startpositionx,startpositiony = pyautogui.position()
		for row in image_array:
			xoffset = 0
			alreadydrawing = False
			for value in row:
				if not value.all(): #Need to draw
					if alreadydrawing == True:
						xoffset += offset
					else:
						startline = startpositionx + xoffset
						alreadydrawing = True
						xoffset += offset
				if value.all(): #End of line aka. white begins
					if alreadydrawing == False:
						xoffset += offset
					else:
						pyautogui.moveTo(startline,startpositiony)
						pyautogui.dragTo(startpositionx + xoffset, startpositiony,duration=lsleep, button = 'left')
						time.sleep(lsleep)
						alreadydrawing = False
						xoffset += offset
			if not value.all():
				if alreadydrawing == True:
					pyautogui.moveTo(startline, startpositiony)
					pyautogui.dragTo(startpositionx + xoffset, startpositiony, duration=lsleep, button ='left')
					time.sleep(lsleep)
					xoffset += offset
			startpositiony += offset
			time.sleep(rsleep)
			

================================================
FILE: src/mouse_automate_color.py
================================================
import numpy
import pyautogui
import time

def closest_color(color, palette):

	r1, g1, b1 = color
	if r1 == 255 and g1 == 105 and b1 == 180:#Failsafe for last color drawn
		cc = r1, g1, b1
		return cc # (255, 105, 180)
	closest_dis = 10000
	cc = 0, 0, 0
	for r2, g2, b2 in palette:
		dis = ((r1 - r2) ** 2 + (g1 - g2) ** 2 + (b1 - b2) ** 2) ** 0.5
		if dis < closest_dis:
			closest_dis = dis
			cc = r2, g2, b2
	return cc
	

def change_color(color,app):
	if color == (255, 105, 180): #Failsafe, needs to be reworked, this should only be temporary
		i = 0
	else:
		i = palette[app][0].index(color)
		
	if app == "paint": 
		"""
		This system to change colors is suboptimal, as different screens have the buttons at different locations! 
		Using json data storage to store these would be better as different subroutines could access them, for example a color mapper.
		"""
		if i <= 9:
			x = i*22 + 878	  # Top row
			y = 58
		else:
			x = (i-10)*22 + 878	  # Bottom row
			y = 82
	else:
		raise ValueError("No color data found for application {}!".format(app))
	pyautogui.mouseUp()
	pyautogui.click(x=x, y=y)

# All color buttons for paint from left to right, top to bottom.
paints = ((0, 0, 0),(127,127,127),(136,0,21),(237,28,36),(255,127,39),(255,242,0),(34,177,76),(0,162,232),(63,72,204),(163,73,164),
		  (255, 255, 255),(195,195,195),(185,122,87),(255,174,201),(255,201,14),(239,228,176),(181,230,29),(153,217,234),(112,146,190),(200,191,231))	
		
palette = {"paint": [paints,paints[10]]}	# Key is the application name which is specified in main.py
		  
class Mouseomate(object):
	pyautogui.PAUSE = 0.00 #useful
	pyautogui.FAILSAFE = True

	@staticmethod
	def image_to_lines(image_array: numpy.array, offset:int, rsleep:int, lsleep:int, app:str) -> None:
		"""
		Converts an image array to mouseclicks.
		:param image_array:
		A numpy array of bools, where False represents a click, and True represents no click.
		:param offset:
		An int which provides spacing between each pixel in image_array. Usefull to adjust for brush size used in whatever this will be outputting for.
		:param rlseep:
		int which designates how long in second the mouse will take drawing a line
		:param rsleep:
		int which designates how long in seconds to pause at end of row
		:return:
		"""
		startpositionx,startpositiony = pyautogui.position()
		for row in image_array:
			xoffset = 0
			white = palette[app][1]
			alreadydrawing = [white,0,0] # Color, starting xoffset, offsets in that line
			row[-1] = [255,105,180] #Failsafe for last color drawn
			# Without it, the last color in a row would not be drawn as the script would skip it, needs fixing!
			for value in row:	
				closest = closest_color(value,palette[app][0])
				if closest == white:
					alreadydrawing[0] = closest
					xoffset += offset
					continue
					
				if alreadydrawing[0] == closest: # Is the current color the same color as the last one checked?
					alreadydrawing[2] += offset 
					xoffset += offset
					continue
			
				alreadydrawing[0] = closest # Define new color

				# Draw line with old color 
				pyautogui.mouseUp()
				pyautogui.moveTo(startpositionx + alreadydrawing[1], startpositiony) #Draw the full line of the last color
				pyautogui.mouseDown()
				pyautogui.dragTo(startpositionx + alreadydrawing[1] + alreadydrawing[2], startpositiony, duration=lsleep, button = 'left')
				pyautogui.mouseUp()
				
				# Define new values for next line
				alreadydrawing[1] = xoffset
				alreadydrawing[2] = 0
				
				# Change the color to the new color
				change_color(closest,app)
					
				time.sleep(lsleep)
				xoffset += offset

			pyautogui.mouseUp()
			change_color(white,app)
			startpositiony += offset
			pyautogui.mouseUp()
			pyautogui.moveTo(startpositionx, startpositiony)
			time.sleep(rsleep)
			# Might have overdone is with mouseUp(), but paint had some problems with recognizing the mouse and handeled it as if it were still drawing, creating some... interesting replications