Cartoon yourself using Leptonica

Intro

I’m a big fan of image processing that I’ve put to use in various previous projects, related to robotics and artificial vision, such as this one, or for video surveillance and motion detection, but also other work projects.
Image processing is a vast and captivating domain, handling ways of image representation and the transformations that can be applied , for unlimited applications. We can use it to extract characteristics related to the image: shapes (such as detecting human faces), numbers and text (for OCR purposes), enhancing the visibility, content readability or simply for aesthetic purposes (in photography), we can encode information (cryptography/security), or make a robot follow the user’s red jacket. The imagination is the limit.
All the image related operations can consume a lot of processing power, so optimized algorithms taking advantage of the way the image is encoded, and doing low level operations at bit level, must be used. Math is also required as many image transformations use mathematical models that not only need to work correctly, but must be implemented in the most optimum way.

Leptonica

Over the time, mostly for academic purposes I’ve implemented various libraries to handle image processing both for mobile and desktop platforms. But recently I decided to use an open source library, named Leptonica, and presented as an “a pedagogically-oriented open source site containing software that is broadly useful for image processing and image analysis applications.” A few days after using it I couldn’t go without it. Not only it offers a robust interface to the basics: various file formats, loading and saving images, supporting and converting from multiple representation systems, but it implements powerful primitives to allow almost any possible transformation on an image. It is open source and so it exposes the internal secrets to anyone willing ot customize the code or to push the performance even further. And talking about performance , it is doing great: bit-wise operations, mathematically improved algorithms, to minimize the work load and do the job in the shortest time.

A sample

As per the article’s title, the plan is to implement a simple cartoon transformation effect, using leptonica, on a regular picture. Well not quite regular, as I chose a beautiful lady for the viewer’s delight.

`PIX *pixSrcImg = pixRead("girl_with_roses.jpg");`

Step 2: convert to grayscale, taking the maximum deviation into account

```PIX *pixGrayMM = pixConvertRGBToGrayMinMax(pixSrcImg, L_CHOOSE_MAX);
pixWrite("leptonica_cartoonizer_01_gray.jpg", pixGrayMM, IFF_JFIF_JPEG);```

Step 3: perform edge detect

```PIX *pixEdges = pixEdgeFilter(pixGrayMM,  L_ALL_EDGES);
pixWrite("leptonica_cartoonizer_02_edges.jpg", pixEdges, IFF_JFIF_JPEG);```

Step 4: invert the edges

```PIX *pixEdgesInverted = pixInvert(NULL, pixEdges);
pixWrite("leptonica_cartoonizer_03_edges_inverted.jpg", pixEdgesInverted, IFF_JFIF_JPEG);```

Step 5: blur the inverted edges, to reduce the roughness a little bit

```PIX *pixBlur = pixBlur5(pixEdgesInverted);
pixWrite("leptonica_cartoonizer_04_blur.jpg", pixBlur, IFF_JFIF_JPEG);```

Step 6: combine the last result with the original image, using a factor (150, can be changed)

```PIX *pixCartoon = pixBlendHardLight(NULL,  pixSrcImg, pixBlur,0,0 ,150);
pixWrite("leptonica_cartoonizer_05_cartoon.jpg", pixCartoon, IFF_JFIF_JPEG);```

The Edge-detect algorithm can be implemented as following:

```/*int matrix_h = {1,0,-1,2,0,-2,1,0,-1}, //sobel
matrix_v = {1,2,1,0,0,0,-1,-2,-1};*
int matrix_h = {-1,-1,-1,2,2,2,-1,-1,-1},
matrix_v = {-1,2,-1,-1,2,-1,-1,2,-1}; //canny
*/
int matrix_h = {1,0,-1,60,0,-60,1,0,-1},
matrix_v = {1,60,1,0,0,0,-1,-60,-1};

PIX * pixEdgeFilter(PIX     *pixs,l_int32  orientflag)
{
l_int32    w, h, d, i, j, wplt, wpld, gx, gy, vald;
l_int32    val1, val2, val3, val4, val5, val6, val7, val8, val9;
PIX       *pixt, *pixd;

PROCNAME("pixSobelEdgeFilter");

if (!pixs)
return (PIX *)ERROR_PTR("pixs not defined", procName, NULL);
pixGetDimensions(pixs, &w, &h, &d);
if (d != 8)
return (PIX *)ERROR_PTR("pixs not 8 bpp", procName, NULL);
if (orientflag != L_HORIZONTAL_EDGES && orientflag != L_VERTICAL_EDGES &&
orientflag != L_ALL_EDGES)
return (PIX *)ERROR_PTR("invalid orientflag", procName, NULL);

/* Add 1 pixel (mirrored) to each side of the image. */
if ((pixt = pixAddMirroredBorder(pixs, 1, 1, 1, 1)) == NULL)
return (PIX *)ERROR_PTR("pixt not made", procName, NULL);

/* Compute filter output at each location. */
pixd = pixCreateTemplate(pixs);
datat = pixGetData(pixt);
wplt = pixGetWpl(pixt);
wpld = pixGetWpl(pixd);
for (i = 0; i < h; i++) {
linet = datat + i * wplt;
lined = datad + i * wpld;
for (j = 0; j < w; j++) {
if (j == 0) {  /* start a new row */
val1 = GET_DATA_BYTE(linet, j);
val2 = GET_DATA_BYTE(linet + wplt, j);
val3 = GET_DATA_BYTE(linet + 2 * wplt, j);
val4 = GET_DATA_BYTE(linet, j + 1);
val5 = GET_DATA_BYTE(linet + wplt, j + 1);
val6 = GET_DATA_BYTE(linet + 2 * wplt, j + 1);
val7 = GET_DATA_BYTE(linet, j + 2);
val8 = GET_DATA_BYTE(linet + wplt, j + 2);
val9 = GET_DATA_BYTE(linet + 2 * wplt, j + 2);
} else {  /* shift right by 1 pixel; update incrementally */
val1 = val4;
val2 = val5;
val3 = val6;
val4 = val7;
val5 = val8;
val6 = val9;
val7 = GET_DATA_BYTE(linet, j + 2);
val8 = GET_DATA_BYTE(linet + wplt, j + 2);
val9 = GET_DATA_BYTE(linet + 2 * wplt, j + 2);
}
if (orientflag == L_HORIZONTAL_EDGES)
vald = L_ABS(	matrix_h*val1 + matrix_h*val2 + matrix_h*val3 +
matrix_h*val4 + matrix_h*val5 + matrix_h*val6 +
matrix_h*val7 + matrix_h*val8 + matrix_h*val9) >> 3;
else if (orientflag == L_VERTICAL_EDGES)
vald = L_ABS(	matrix_v*val1 + matrix_v*val2 + matrix_v*val3 +
matrix_v*val4 + matrix_v*val5 + matrix_v*val6 +
matrix_v*val7 + matrix_v*val8 + matrix_v*val9) >> 3;
else {  /* L_ALL_EDGES */
gx = L_ABS(	matrix_v*val1 + matrix_v*val2 + matrix_v*val3 +
matrix_v*val4 + matrix_v*val5 + matrix_v*val6 +
matrix_v*val7 + matrix_v*val8 + matrix_v*val9) >> 3;

gy = L_ABS(	matrix_h*val1 + matrix_h*val2 + matrix_h*val3 +
matrix_h*val4 + matrix_h*val5 + matrix_h*val6 +
matrix_h*val7 + matrix_h*val8 + matrix_h*val9) >> 3;
vald = L_MIN(255, gx + gy);
}
SET_DATA_BYTE(lined, j, vald);
}
}

pixDestroy(&pixt);
return pixd;
}
```

1. Mamad says:

Thanks for excelent post.

I’ve implemented this approach but the final result is not good.
May I know which edge detection algorithm you are using please? I’m using Sobel:

PIX *edges = pixSobelEdgeFilter(gray, L_ALL_EDGES);
and there are less edges compared to step 3 image.

Also, I’ve used this kernel for bluring the inverted edge image:

static const char* BLUR_KERNEL = ” 5 10 20 10 5 ” ” 10 20 50 20 10 ” ” 10 70 140 70 10 ” ” 10 20 50 20 10 ” ” 5 10 20 10 5 “;
L_KERNEL* bKernel = kernelCreateFromString(5,5,2,2,BLUR_KERNEL);
PIX* blur = pixConvolve(inverted, bKernel, 8, 1);

Thanks

2. Radu Motisan says:

3. Radu Motisan says:
4. Mamad says: