diff --git a/docs/pic/robot_hand_sideView.pdf b/docs/pic/robot_hand_sideView.pdf new file mode 100644 index 0000000..f03d42a Binary files /dev/null and b/docs/pic/robot_hand_sideView.pdf differ diff --git a/docs/pic/robot_hand_sideView.svg b/docs/pic/robot_hand_sideView.svg new file mode 100644 index 0000000..493f8d1 --- /dev/null +++ b/docs/pic/robot_hand_sideView.svg @@ -0,0 +1,4177 @@ + + + +bPMF1F21Dept.Technical referenceCreated byApproved byDocument typeDocument statusTitleDWG No.Rev.Date of issueSheet29.04.20261/1Assembly_SchlankChristoph Kendel diff --git a/docs/pic/robot_hand_sideView_0.pdf b/docs/pic/robot_hand_sideView_0.pdf new file mode 100644 index 0000000..28f4d1b Binary files /dev/null and b/docs/pic/robot_hand_sideView_0.pdf differ diff --git a/docs/pic/robot_hand_topView.pdf b/docs/pic/robot_hand_topView.pdf new file mode 100644 index 0000000..41d12e4 Binary files /dev/null and b/docs/pic/robot_hand_topView.pdf differ diff --git a/docs/pic/robot_hand_topView.svg b/docs/pic/robot_hand_topView.svg new file mode 100644 index 0000000..d5a8414 --- /dev/null +++ b/docs/pic/robot_hand_topView.svg @@ -0,0 +1,4652 @@ + + + +MF1F21Dept.Technical referenceCreated byApproved byDocument typeDocument statusTitleDWG No.Rev.Date of issueSheet29.04.20261/1Assembly_SchlankChristoph Kendel diff --git a/docs/pic/robot_hand_topView_0.pdf b/docs/pic/robot_hand_topView_0.pdf new file mode 100644 index 0000000..f5c220c Binary files /dev/null and b/docs/pic/robot_hand_topView_0.pdf differ diff --git a/docs/pic/robot_image_b.png b/docs/pic/robot_image_b.png new file mode 100644 index 0000000..accfe5f Binary files /dev/null and b/docs/pic/robot_image_b.png differ diff --git a/docs/pic/robot_image_c.png b/docs/pic/robot_image_c.png new file mode 100644 index 0000000..1a0b9a5 Binary files /dev/null and b/docs/pic/robot_image_c.png differ diff --git a/docs/pic/robot_image_d.png b/docs/pic/robot_image_d.png new file mode 100644 index 0000000..b24363e Binary files /dev/null and b/docs/pic/robot_image_d.png differ diff --git a/docs/pic/robot_sideView_forearm.pdf b/docs/pic/robot_sideView_forearm.pdf index 8fbf402..61c626e 100644 Binary files a/docs/pic/robot_sideView_forearm.pdf and b/docs/pic/robot_sideView_forearm.pdf differ diff --git a/docs/pic/robot_sideView_forearm.svg b/docs/pic/robot_sideView_forearm.svg index 7c5d503..0f1cebf 100644 --- a/docs/pic/robot_sideView_forearm.svg +++ b/docs/pic/robot_sideView_forearm.svg @@ -24,11 +24,11 @@ inkscape:deskcolor="#d1d1d1" inkscape:document-units="mm" inkscape:zoom="2.2299458" - inkscape:cx="307.18235" - inkscape:cy="123.76982" + inkscape:cx="282.51808" + inkscape:cy="85.203865" inkscape:window-width="1858" inkscape:window-height="826" - inkscape:window-x="58" + inkscape:window-x="46" inkscape:window-y="247" inkscape:window-maximized="0" inkscape:current-layer="layer-oc2">242226P250 mm90 mm on input line 22. LaTeX Font Info: External font `lmex10' loaded for size (Font) <6> on input line 22. -LaTeX Font Info: Trying to load font information for T1+lmtt on input line 3 -4. - -(C:\Program Files\MiKTeX\tex/latex/lm\t1lmtt.fd -File: t1lmtt.fd 2015/05/01 v1.6.1 Font defs for Latin Modern -) - -File: pic/robot_image_a.png Graphic file (type png) - -Package pdftex.def Info: pic/robot_image_a.png used on input line 43. -(pdftex.def) Requested size: 360.0pt x 163.77846pt. LaTeX Font Info: External font `lmex10' loaded for size -(Font) <10.95> on input line 48. -LaTeX Font Info: Trying to load font information for TS1+lmr on input line 5 -4. - -(C:\Program Files\MiKTeX\tex/latex/lm\ts1lmr.fd -File: ts1lmr.fd 2015/05/01 v1.6.1 Font defs for Latin Modern -) -Underfull \hbox (badness 2293) in paragraph at lines 55--57 -\T1/lmr/m/n/10.95 (each po-si-ti-on on its - [] - - -File: pic/robot_sideView_measurements.pdf Graphic file (type pdf) - -Package pdftex.def Info: pic/robot_sideView_measurements.pdf used on input lin -e 62. -(pdftex.def) Requested size: 216.0022pt x 155.8966pt. +(Font) <10.95> on input line 37. [1 {C:/Users/kech/AppData/Local/MiKTeX/fonts/map/pdftex/pdftex.map}{C:/Program Fil -es/MiKTeX/fonts/enc/dvips/lm/lm-ec.enc}{C:/Program Files/MiKTeX/fonts/enc/dvips -/lm/lm-mathsy.enc} <./pic/robot_image_a.png>] +es/MiKTeX/fonts/enc/dvips/lm/lm-ec.enc}] +LaTeX Font Info: Trying to load font information for T1+lmtt on input line 4 +6. + +(C:\Program Files\MiKTeX\tex/latex/lm\t1lmtt.fd +File: t1lmtt.fd 2015/05/01 v1.6.1 Font defs for Latin Modern +) + +File: pic/robot_image_a.png Graphic file (type png) + +Package pdftex.def Info: pic/robot_image_a.png used on input line 56. +(pdftex.def) Requested size: 360.0pt x 163.77846pt. +LaTeX Font Info: Trying to load font information for TS1+lmr on input line 7 +1. + +(C:\Program Files\MiKTeX\tex/latex/lm\ts1lmr.fd +File: ts1lmr.fd 2015/05/01 v1.6.1 Font defs for Latin Modern +) +Underfull \hbox (badness 2293) in paragraph at lines 72--74 +\T1/lmr/m/n/10.95 (each po-si-tion on its + [] + + +File: pic/robot_sideView_measurements.pdf Graphic file (type pdf) + +Package pdftex.def Info: pic/robot_sideView_measurements.pdf used on input lin +e 79. +(pdftex.def) Requested size: 216.0022pt x 155.8966pt. + + +[2{C:/Program Files/MiKTeX/fonts/enc/dvips/lm/lm-mathit.enc}{C:/Program Files/M +iKTeX/fonts/enc/dvips/lm/lm-rm.enc}{C:/Program Files/MiKTeX/fonts/enc/dvips/lm/ +lm-mathsy.enc}{C:/Program Files/MiKTeX/fonts/enc/dvips/lm/lm-ts1.enc} <./pic/ro +bot_image_a.png> <./pic/robot_sideView_measurements.pdf>] LaTeX Font Info: Trying to load font information for OT1+lmtt on input line -86. +103. (C:\Program Files\MiKTeX\tex/latex/lm\ot1lmtt.fd File: ot1lmtt.fd 2015/05/01 v1.6.1 Font defs for Latin Modern ) - + File: pic/robot_frontView_forearm.pdf Graphic file (type pdf) -Package pdftex.def Info: pic/robot_frontView_forearm.pdf used on input line 99 -. +Package pdftex.def Info: pic/robot_frontView_forearm.pdf used on input line 11 +4. (pdftex.def) Requested size: 132.4757pt x 108.0011pt. - + File: pic/robot_sideView_forearm.pdf Graphic file (type pdf) -Package pdftex.def Info: pic/robot_sideView_forearm.pdf used on input line 100 +Package pdftex.def Info: pic/robot_sideView_forearm.pdf used on input line 115 . (pdftex.def) Requested size: 245.50255pt x 108.0011pt. -Overfull \hbox (28.94739pt too wide) in paragraph at lines 98--101 -[] [] +Overfull \hbox (27.11282pt too wide) in paragraph at lines 113--116 +[]| [] [] [] +LaTeX Font Info: External font `lmex10' loaded for size +(Font) <9> on input line 148. +LaTeX Font Info: External font `lmex10' loaded for size +(Font) <5> on input line 148. -[2{C:/Program Files/MiKTeX/fonts/enc/dvips/lm/lm-ts1.enc}{C:/Program Files/MiKT -eX/fonts/enc/dvips/lm/lm-rm.enc}{C:/Program Files/MiKTeX/fonts/enc/dvips/lm/lm- -mathit.enc}{C:/Program Files/MiKTeX/fonts/enc/dvips/lm/lm-rmtt.enc} <./pic/robo -t_sideView_measurements.pdf> <./pic/robot_frontView_forearm.pdf - -pdfTeX warning: miktex-pdftex.exe (file ./pic/robot_frontView_forearm.pdf): PDF - inclusion: multiple pdfs with page group included in a single page -> <./pic/robot_sideView_forearm.pdf +[3{C:/Program Files/MiKTeX/fonts/enc/dvips/lm/lm-rmtt.enc} <./pic/robot_frontVi +ew_forearm.pdf> <./pic/robot_sideView_forearm.pdf pdfTeX warning: miktex-pdftex.exe (file ./pic/robot_sideView_forearm.pdf): PDF inclusion: multiple pdfs with page group included in a single page >] + +File: pic/robot_hand_sideView.pdf Graphic file (type pdf) + +Package pdftex.def Info: pic/robot_hand_sideView.pdf used on input line 155. +(pdftex.def) Requested size: 156.53336pt x 108.0011pt. + +File: pic/robot_hand_topView.pdf Graphic file (type pdf) + +Package pdftex.def Info: pic/robot_hand_topView.pdf used on input line 157. +(pdftex.def) Requested size: 111.12631pt x 93.59802pt. -[3] (position.aux) + +[4 <./pic/robot_hand_sideView.pdf> <./pic/robot_hand_topView.pdf + +pdfTeX warning: miktex-pdftex.exe (file ./pic/robot_hand_topView.pdf): PDF incl +usion: multiple pdfs with page group included in a single page +>] (position.aux) *********** LaTeX2e <2024-11-01> patch level 2 L3 programming layer <2025-04-29> *********** ) Here is how much of TeX's memory you used: - 2992 strings out of 469923 - 50836 string characters out of 5479241 - 436765 words of memory out of 5000000 - 29782 multiletter control sequences out of 15000+600000 - 662462 words of font info for 67 fonts, out of 8000000 for 9000 + 1877 strings out of 469923 + 30745 string characters out of 5479241 + 432319 words of memory out of 5000000 + 28679 multiletter control sequences out of 15000+600000 + 685554 words of font info for 81 fonts, out of 8000000 for 9000 1141 hyphenation exceptions out of 8191 - 57i,6n,65p,227b,236s stack positions out of 10000i,1000n,20000p,200000b,200000s - -Output written on position.pdf (3 pages, 1427690 bytes). + 57i,8n,65p,227b,261s stack positions out of 10000i,1000n,20000p,200000b,200000s + +Output written on position.pdf (4 pages, 1766750 bytes). PDF statistics: - 2089 PDF objects out of 2487 (max. 8388607) + 3090 PDF objects out of 3580 (max. 8388607) 0 named destinations out of 1000 (max. 500000) - 21 words of extra memory for PDF output out of 10000 (max. 10000000) + 31 words of extra memory for PDF output out of 10000 (max. 10000000) diff --git a/docs/position.pdf b/docs/position.pdf index d075abf..c2d9663 100644 Binary files a/docs/position.pdf and b/docs/position.pdf differ diff --git a/docs/position.synctex.gz b/docs/position.synctex.gz index ba2d3a2..f97f13b 100644 Binary files a/docs/position.synctex.gz and b/docs/position.synctex.gz differ diff --git a/docs/position.tex b/docs/position.tex index 11c72e7..abdca1c 100644 --- a/docs/position.tex +++ b/docs/position.tex @@ -8,12 +8,12 @@ \usepackage{graphicx} % Deutsche Sprache (Datum, Trennungen, etc.) -\usepackage[ngerman]{babel} +%\usepackage[ngerman]{babel} % Optional: bessere Schrift \usepackage{lmodern} -\title{How to get a robot position from video image} +\title{How to get a robot position from video images} \author{Christoph Kendel} \date{\today} @@ -21,32 +21,49 @@ \maketitle -\section{Intro} +\begin{center} +\begin{minipage}{0.89\linewidth} +\section{Info} -My Robot-Arm does not have homing switches. I want to use WebCams and Aruco--Markers to get the position of each joint. This way -I can get the position as well as double--check the position while working under load. +My Robot-Arm does not have homing switches. I use WebCams to get the position of each joint. This way +I can get the initial position without movement as well as contiuously check the position while working under load. -I work with Arucos. With my low resolution WebCam I can get reliable positions (when using two cams). But the angle resolution is not usable. -So I have to calculate the angle of each element from the relative positions of the markers. +\vspace{2mm} +\hspace{5mm} +I work with Aruco markers. With budget low resolution WebCams I can get reliable positions (when using two cams). But the angular recognizion of the markers turned out +not to be reliable. Thus + I have to calculate the angle of each element from the relative positions of the markers. Here I explain how I these calculations are done. +\end{minipage} +\end{center} + +\pagebreak \section{Angles} +The linear $x$ Position is easy to get: the markers 201, 204, 200, 198, 229, 243 have all a fixed relative $x$ position. Thus the average +can be taken, wich is saved as $x_{242}$ the x-position of the hand-rotation-joint. + +All the other values are the angles {\tt y, z, a, b, c} and {\tt e}. % (e for the opening-of the hand). + \subsection{Bizeps {\bf \tt y}} To find the angle of the bizeps (upper arm) there are different options, depending on the angle-of-view and which ArUcos are visable. -\vspace{2mm} -\noindent +\begin{figure}[h!] \includegraphics[width=\linewidth]{pic/robot_image_a.png} +\caption{Video Capture with recognized Aruco markers} +\end{figure} -\vspace{2mm} \noindent The calculation is based on two approaches: Position of the Markers 243 etc on the one hand, and relative position 198 $\leftrightarrow$ 229 on the other hand. + +\vspace{2mm} + \noindent \begin{minipage}{0.39\linewidth} Several ways to calculate {\tt y}: @@ -79,11 +96,11 @@ At the ellbow there is a motor that turns the forearm arround. Depending on whic % the {\tt y} and {\tt z} position of 223 etc can't be used, as it depends on the angle of the forearm and the angle of the bizeps. Thus is it less reliable. (Although it would be a nice -thing to use as a double--check). With the $x_{223}$ compared to the $x_{242}$ i can calculate the ellbow turning {\tt a} value: +thing to use as a double--check). With the $x_{223}$ compared to the $x_{226}$ i can calculate the ellbow turning {\tt a} value: $$ -\sin({\tt a} + \alpha_{\rm }) = \frac{\Delta x}{35\,\rm mm} = \frac{x_{223} - x_{242}}{35\,\rm mm} +\sin({\tt a} + \alpha_{\rm }) = \frac{\Delta x}{35\,\rm mm} = \frac{x_{223} - x_{226}}{35\,\rm mm} $$ @@ -92,14 +109,12 @@ as I know the angle $\alpha$ (yellow triangle) from the printed geometry of the sled and bizeps, as their markers are fixed in $x$ position. This $\alpha$ is calculated for all markers of the forearm. -\vspace{2mm} - - -\noindent -\includegraphics[height=0.3\linewidth]{pic/robot_frontView_forearm.pdf}\hspace{3mm} +\begin{figure}[h!] +\hspace*{-4.5mm} +\includegraphics[height=0.3\linewidth]{pic/robot_frontView_forearm.pdf}\hspace{6mm} \includegraphics[height=0.3\linewidth]{pic/robot_sideView_forearm.pdf} - -\vspace{2mm} +\caption{Front view and sideview of the forearm with calculation variables.} +\end{figure} \noindent As the rotation {\tt a} may change for different markers, the median is taken. @@ -107,10 +122,57 @@ As the rotation {\tt a} may change for different markers, the median is taken. \subsection{Forearm} -With the +The position of the forearm motor (controlling motor in the bizeps) can be calculated from the difference between the bizeps +position $y$ and the angle of the forearm. Thus we need the forearm--angle. That can be calculated in two ways +% +% +\begin{itemize} +\item relative position of two markers on the forearm, that are on one line. +\item relative $y, z$ position of one marker on the forearm and one marker on the ellbow, e.g. 242 or 222. +\end{itemize} +% +Again: +As I might get different results, I have to find a median value. +\vspace{2mm} + +\hfill $\Rightarrow$ Hand-Joint $P$ is known. + +\subsection{Hand} + +To get the hand position, the easiest way I see is to calculate the middle position between both finger-markers% +\footnote{The +other way to calculate the hand position without the markers orientation would be: calculate the middle-hand-plane from +the forearm rotation, with the distance of the finger-point to that plane, you'd also get the finger-opening and the projectet position of the finger-marker +to that plane, you'd get the angle.} +($F_1$, $F_2$ here shown in green). +The direction $P \to M$ in combination with the forearm-direction results in the angle {\tt b}. +\begin{figure}[h!] +\hfill +\includegraphics[height=0.3\linewidth]{pic/robot_hand_sideView.pdf} +\hfill +\includegraphics[height=0.26\linewidth]{pic/robot_hand_topView.pdf} +\hfill {} +\caption{\small Hand with points to calculate the values for {\tt b, c, e}.} +\end{figure} + + +\noindent +The Distance between the hand markers is a result of the opening of the fingers {\tt e}. +So I can calculate {\tt e} from that distance. + +Furthermore the direction of the distance between the grippers gives me the rotation of the hand {\tt c}. + +\paragraph{Limitations with the Hand:} The results depend heavily on the accuracy of $P$ and $M$. Both are never measured directly. So have to +double check if they make sense. Only if the distance $\overline {PM}$ checks up to be the mechanical correct, that can be used to evaluate the correctnes of $P$ and $M$. +And only if the point $M$ lies in the plane of the rotation {\tt a} arround the forearm. + + +\section{Program} + +The NodeJS Program calculates the angles and checks, if they are the ones at the moment represented by the system. \end{document} \ No newline at end of file