How to detect square which has a specific width value in images and check the intersection of 2 lines within the circle?

Question

I want to use the Python OpenCV library to perform image processing and detect squares in an image. The squares have a certain width, but the specific width value is unknown. Then I want to draw a circle with the center of the square as the center point and a radius equal to four times the distance from the center to the square's contour. There are two intersecting lines in the image, and their positions are not fixed. The clarity of the lines may vary in different images. You can notice that there are many noise dots on the image, and the brightness levels are also inconsistent with different images. How can I determine if the intersection point of the two lines falls within the circle? Red arrow indicates the intersection point of 2 lines in the below image.

I used 4 images for testing. Currently, I convert the color image to grayscale, apply Gaussian blur, use the Canny algorithm for edge detection, and then perform contour detection. The problem I'm facing is that due to the width of the square, some images have the square contour drawn outside the square, some have it drawn inside, and some have both inside and outside contours. How can I update the algorithm to only draw the outer contour? I have already tried adjusting the values in cv2.Canny(blur, 50, 100), but it's difficult to find a set of values that work for all images. And cv2.findContours(edges, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE),if change cv2.RETR_TREE to cv2.EXTERNAL, it cannot find the center point and circle.

Looking forward to receiving some useful suggestions from you.

The 4 images for testing are as below.

Pic_1: Pic_2: Pic_3: Pic_4:

My Python code is as below.

import cv2
import glob
import os
import numpy as np

def detect_squares(img_path):
    
    file_name = os.path.basename(img_path)
    
    # Read the image and create a copy for drawing results
    img = cv2.imread(img_path)
    image_with_rectangles = img.copy()

    # Convert to grayscale
    gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
    # Apply Gaussian blur
    blur = cv2.GaussianBlur(gray, (5, 5), 0)
    # Perform edge detection
    edges = cv2.Canny(blur, 50, 100)

    # Find contours
    contours, _ = cv2.findContours(edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)

    # Traverse through the detected contours
    for contour in contours:
        # Calculate contour perimeter
        perimeter = cv2.arcLength(contour, True)
        # Get the coordinates of contour vertices
        approx = cv2.approxPolyDP(contour, 0.04 * perimeter, True)
        # Get the coordinate values, width, and height
        x, y, w, h = cv2.boundingRect(approx)
        # Classify the contour
        if len(approx) == 4 and perimeter >= 350 and 0.9 <= float(w) / h <= 1.1:

            # Draw red contour lines
            cv2.drawContours(image_with_rectangles, contour, -1, (0, 0, 255), cv2.FILLED)
    
            # Calculate contour moments
            M = cv2.moments(contour)

            # Calculate the centroid coordinates of the contour
            if M["m00"] != 0:
                center_x = int(M["m10"] / M["m00"])
                center_y = int(M["m01"] / M["m00"])
                center = (center_x, center_y)

                # Draw the center point on the image
                cv2.circle(image_with_rectangles, center, 5, (0, 255, 255), 3)

                # Calculate the distance from the center to the square contour
                distance = int(cv2.pointPolygonTest(approx, center, True))

                # Calculate the radius of the circle
                radius = 4 * distance

                # Draw the circle
                cv2.circle(image_with_rectangles, (center_x, center_y), radius, (0, 255, 255), 2)

    # Show the resulting image
#     cv2.imshow(f"Canny Detection - {file_name}", edges)
    cv2.imshow(f"Shape Detection - {file_name}", image_with_rectangles)

    cv2.waitKey(0)
    cv2.destroyAllWindows()
    
detect_squares(image_path)

Please share your 6 images - without yellow annotations - so we can be aware of any likely issues or edge cases. Thank you. — Mark Setchell, Jul 15 '23 at 08:26
those images still have annotation in them: the crosshairs. as such, they are illustrations, not data. if you expect anyone to run your code, you need to provide data. — Christoph Rackwitz, Jul 15 '23 at 09:13
@ChristophRackwitz the data for running the code is just the images in the question description — Ringo, Jul 15 '23 at 09:37
so you expect image processing to be done on an image with long red lines through it? those lines break image content. you need to figure out how to get clean feed images, not those ***manually cropped screenshots*** off of whatever video display your pick-and-place machine gives you. — Christoph Rackwitz, Jul 15 '23 at 09:42
in the real case, there are 2 intersecting lines on the picture, the color and locations are different from the ones on the image.I need to check whether the intersection in or out of the yellow circle. i will use this Python function to check for thousands of images. — Ringo, Jul 15 '23 at 09:55
Canny is hardly ever the best solution where you have clear objects with a high contrast. And it is hardly ever the right solution where you want to detect lines. I can imagine just thresholding the image to find the black shapes, then filter them by shape to find the square one. — Cris Luengo, Jul 15 '23 at 13:11
those aren't wires. they're drawn by a program, both the orange lines and the shadows. these pictures are screenshots, manually cropped. check the sizes. the fuzziness is due to SO deciding to store them either as PNG or JPEG, by some opaque rules. these lines disturb any image processing. the very first picture: one line divides the square. the 'center" point is the center of the top 90% of the divided square. these lines need to be dealt with. — Christoph Rackwitz, Jul 15 '23 at 13:19
I'm sorry for any confusion caused. In fact, the images are automatically taken by a camera with a fixed format, so there won't be a situation where one photo is in JPEG and another is in PNG. Also, the photos will have consistent sizes. The crosshair lines shown in the images are automatically created by APP. In reality, the images to be processed will not have those two orange lines. Instead, there will be two gray lines, but their positions may vary (they may not be horizontal or vertical). The above 6 images are screenshots. — Ringo, Jul 15 '23 at 14:06
There are currently two main issues: 1) How can we accurately draw the circle on the image? The current approach involves having both an inner and outer square contour, which results in an inaccurate radius for the circle. Additionally, the program might automatically draw two circles, like the picture in the above question description. 2) Another one is how to determine whether the intersection point of two lines is inside or outside the drawn circle. What kind of algorithm can effectively locate the intersection point when the two lines in the actual image may not be very clear." — Ringo, Jul 15 '23 at 14:06
I have uploaded an image that needs to be processed in the question description. But it's still a screenshot, just to illustrate what the actual two lines on the image that needs to be processed look like. — Ringo, Jul 15 '23 at 14:24
Ok, if you don’t have an actual image that your software will have to process, you need to stop writing code and acquire some of those images. Screen shots and annotated example images are useless when you’re developing an image analysis algorithm. Get some data in exactly the way that your program will process when in production. Anything else is a waste of time. — Cris Luengo, Jul 15 '23 at 15:18
@CrisLuengo Already upload 4 actual images in the question description. Can you please give some suggestions? — Ringo, Jul 16 '23 at 02:12
Those are significantly different from the images you shared earlier. Indeed, any solutions for those earlier images would have been useless. — Cris Luengo, Jul 16 '23 at 02:59
A top hat will isolate the markings, and a threshold after that will give you a good, clear set of objects. Select the object that is most square (eg by looking at the size of the minimum bounding box), its centroid and size are what you seek. Next find lines radially emanating approximately from the middle, by detecting lines and selecting for orientation. This should isolate the two orthogonal lines, and give you their intersection point. Developing each of these steps is not trivial, but shouldn’t be too complex either. — Cris Luengo, Jul 16 '23 at 03:03
how come these new images are vignetted while the previous ones were not? do not apply any effects to these pictures. — Christoph Rackwitz, Jul 16 '23 at 13:15
these images are the actual with different camera. I can detect the square and draw the circle now, but now the problem is that how to detect the 2 lines which have the intersection. For me it's difficult. — Ringo, Jul 16 '23 at 13:38

Mark Setchell · Answer 1 · 2023-07-16T19:35:58.457

As the square is the only fully-enclosed shape in the image (apart from the zeroes in the numbers), they are the only place where, if you do a flood-fill, you will not fill (almost) the entire image. So you could start from the centre of the image, or maybe the brightest spot, and do flood-fills till you get the right shape/size of flood-filled area. You don't need to seed the flood-fill at every single pixel, just use centres at say 80% of the square's side-length. Or if you normalised the lighting, you could fill from an edge and your square would be the only remaining unfilled shape.

So, a division normalisation of the lighting gives this:

I did it with ImageMagick like this:

magick INPUT.JPG \( +clone -blur 0x19 \) +swap -compose divide -composite  -threshold 90% result.jpg

but Fred @fmw42 gives a Python version here.

You can then do a flood-fill - again I am doing it with ImageMagick but OpenCV has a similar function. If you flood-fill with red, starting at [1350,950] you get:

And if you flood-fill from [1300,900] you get:

Note I am only using red for illustration purposes - there is no need to introduce a third channel and triple the memory and processing demand in the real application. As the image is already thresholded to pure black and white, you can use mid-grey or similar for the fill colour.

How to detect square which has a specific width value in images and check the intersection of 2 lines within the circle?

1 Answers1