• AVR Freaks

Helpful ReplyHot!Legato and Issues with Layer Transparency

Author
baugs
New Member
  • Total Posts : 17
  • Reward points : 0
  • Joined: 2009/10/29 13:31:38
  • Location: 0
  • Status: offline
2020/10/15 15:23:30 (permalink)
0

Legato and Issues with Layer Transparency

I am running into problems with Legato in a multi-layer application. The PIC32MZ2064DAR176 that I am using incorporates a graphics LCD controller which supports up to 3 overlay layers that are merged in hardware. Aria, while leaving much to be desired, supported multiple layers. Legato claims to support multiple layers now but, unless I am missing something, there is currently a rather significant flaw/oversight in the implementation. I'll explain this in the hopes that someone has a simple fix that I've missed.

The basic issue is that in its current form (GFX v3.8.1) I can find no straightforward way to set any part of a layer as transparent in Legato.  Not only do I not see a way to initialize a layer as transparent, I can't see how transparency can be maintained in a layer (i.e. set a layer pixel back to transparent after it was non-transparent such as when label text is changed). I know this may seem hard to believe.

Apparently multi-layer support was added only recently in Legato and most of the initial development was done with only a single layer. I think this largely explains why the code was written the way it was, although I can't understand how they could release it in its current form claiming multi-layer support when it has this fundamental limitation which makes it unusable.

Some observations:

- There is no flag or setting to indicate that a layer is transparent and the code does not treat top layers any differently than Layer 0.
- Let's say you have a 3 layer setup. Layer 0 has a panel widget with a solid color background that covers the entire screen area. Layer 0 also contains a single label near the bottom of the screen. Layers 1 and 2 are totally empty (no widgets). Layer 0 will be drawn correctly. Layer 1 will be filled with whatever the contents of the scratch buffer are from the last draw event. Layer 2 will also be filled with garbage. The final image will resemble some portion of the label from Layer 0 copied repeatedly vertically from the top of the screen to the bottom.
- Creating a panel widget on Layers 1 and 2 with a background fill and a 0 alpha background color doesn't solve the problem. If you don't enable alpha blending then the background alpha will default to 0xFF (overriding the alpha value of the color) and layers underneath will be totally obscured. If you set the alpha to 0 then the pixels won't be modified at all as the code skips drawing a widget when the alpha is 0. Even if you modified the code to draw when the widget alpha is 0 it wouldn't help the problem. This is because the underlying pixel data will be whatever junk data resides in the scratch buffer from the last draw. The background would be blended with this, leaving the pixel unchanged.
- This is the best/easiest fix that I can think of at the moment: Change the rendering engine so that any layer that is on top of another layer defaults to being transparent. In Legato, layers have root widgets that are simple leWidgets with no backgrounds. These root widgets should be handled differently by the renderer and result in default pixel values with an alpha of 0 drawn over the entire screen area.

While multi-layers worked in Aria, going back to Aria is not really an option for me. Legato has a lot of performance improvements that, IMO, are particularly important for the PIC32MZ/DA series. In my experience these chips were hamstrung by Aria, especially in demanding applications, due to issues related to GPU usage, DDR bandwidth, etc. The success of the project I'm working on is somewhat dependent on Legato solving these issues. There is a lot to like in the new renderer in Legato otherwise. I've found a good number of bugs which I've been able to work through so far but in the case of layer transparency the feature simply appears to be missing altogether. Before I implement what I described, I just wanted to see if anyone else has encountered this and developed their own solution or knows an easier workaround.
#1
MHGC
Super Member
  • Total Posts : 250
  • Reward points : 0
  • Status: offline
Re: Legato and Issues with Layer Transparency 2020/10/15 16:09:18 (permalink) ☄ Helpfulby baugs 2020/10/16 10:08:31
0
Hi,
 
Here are the steps we had to do:

We will see what we can do to make this more user-friendly in v3.9.0.
 
We will also look to publish some guidelines about GPU usage, avoiding collisions with GLCD and overall traffic management with the DDR bus bandwidth.
 
Can you list the bugs you have fixed in the renderer?  We can look to roll them into the next release.
 
Thank you for your feedback.
 

Attached Image(s)

#2
baugs
New Member
  • Total Posts : 17
  • Reward points : 0
  • Joined: 2009/10/29 13:31:38
  • Location: 0
  • Status: offline
Re: Legato and Issues with Layer Transparency 2020/10/16 13:30:06 (permalink)
5 (1)
Thank you for the quick response and helpful information.

I followed your recommendation and the layer transparency is working for me now. I am a bit confused though, as by my recollection this is exactly one of the permutations that I had tried (with the exception of leaving the enabled property selected, which doesn't appear to make a difference). I already had the clear scheme and had tried it with alpha blending for the panel widget both ON and OFF and using different alpha values. Perhaps I thought I was using my clear scheme but wasn't or maybe it wasn't getting assigned somehow? I have no idea but it works now.

bill.liWe will also look to publish some guidelines about GPU usage, avoiding collisions with GLCD and overall traffic management with the DDR bus bandwidth.


That would be extremely helpful. I am doing a lot of custom drawing and have implemented MSAA, rotatable primitives with subpixel resolution, etc. Up to now I have been determining what works well and what doesn't largely through experimentation. This has slowly given me an understanding of best practices with this architecture but having this knowledge upfront would save a lot of time.

Can you list the bugs you have fixed in the renderer?  We can look to roll them into the next release.


Sure. This is what I have at the moment:

legato_renderer.c

I think there are several problems in postFrame().

line 751

leRectArray_Copy(&_rendererState.layerStates[_rendererState.layerIdx].currentDamageRects,
                         &_rendererState.layerStates[_rendererState.layerIdx].prevDamageRects);               
                        
When this line is executed, _rendererState.layerIdx will be equal to LE_LAYER_COUNT. This will cause an out-of-bounds access as layerStates is of length LE_LAYER_COUNT. Was this line intended to be called for each active layer?

lines 766, 768, 772

Each has the same issue as line 751.

line 786:

if(_rendererState.layerStates[_rendererState.layerIdx].pendingDamageRects.size > MAX_RECTARRAYS_SZ)

I think this should be:

if(_rendererState.layerStates[itr].pendingDamageRects.size > MAX_RECTARRAYS_SZ)

Other questions/comments:

_rendererState.drawCount is only incremented but never reset to 0. How would the first condition on line 764 ever be tripped more than once?

- Is the purpose of prevDamageRects to help sync the double buffer to the changes made in the last draw of the active buffer? If so, isn't this redundant on the PIC32MZ/DA since the syncing is already done in DRV_GLCD_Update() (drv_gfx_glcd.c) with the help of the syncRect's stored in its local drvLayer?

legato_widget_list.c

appendItem()

item->icon is not assigned and could have any value since LE_MALLOC is used to create 'item'. After line 404, I added this line:

item->icon = NULL;

appendItem()

Same issue and fix as appendItem().

setItemIcon()

I think that the list area should be invalidated here. This isn't technically a bug but produces the more expected behavior, IMO. After line 949, I added:

_invalidateListArea(_this);

legato_widget_list_skin_classic.c

leListWidget_GetListRec()

Results should be translated to screen space before returning. After line 131 add:

leUtils_RectToScreenSpace((leWidget*)lst->widget.parent, rect);


Thanks again!
post edited by baugs - 2020/10/16 13:33:23
#3
baugs
New Member
  • Total Posts : 17
  • Reward points : 0
  • Joined: 2009/10/29 13:31:38
  • Location: 0
  • Status: offline
Re: Legato and Issues with Layer Transparency 2020/10/16 13:38:31 (permalink)
0
The second mention of appendItem() in my last post should be insertItem(). It's not letting me edit the post directly.
#4
Ed@Microchip
Super Member
  • Total Posts : 164
  • Reward points : 0
  • Joined: 2017/04/06 09:39:29
  • Location: 0
  • Status: offline
Re: Legato and Issues with Layer Transparency 2020/10/19 09:53:47 (permalink)
0
baugs
with the exception of leaving the enabled property selected, which doesn't appear to make a difference



Hi baugs,
 
Correct, it doesn't make any difference visually.
 
However, unchecking the 'enabled' property on the panel is needed so that touch events can propagate down to the widgets in the lower layers. Since the panel covers the entire upper layer - if enabled, it will catch all touch events (even if it's transparent) and widgets in the lower layers will not receive any touch input. It's something to consider if you interactive widgets, like buttons, in the lower layer(s).
 
Thanks,
 
Ed @ Microchip
#5
baugs
New Member
  • Total Posts : 17
  • Reward points : 0
  • Joined: 2009/10/29 13:31:38
  • Location: 0
  • Status: offline
Re: Legato and Issues with Layer Transparency 2020/10/26 10:46:31 (permalink)
0
bill.liWe will also look to publish some guidelines about GPU usage...


On that note, perhaps someone can help me with a GPU-related question.

I can't figure out how to get Nano2D to use the individual pixel alpha in a blend (N2D_RGBA8888 color mode). No matter what I try, all GPU-accelerated blits ignore pixel alpha unless it is a 1:1 blit, i.e. N2D_BLEND_NONE, in which case the pixel alpha channel from the source is correctly copied to the destination. Surely there is a way to do a GPU-accelerated blit blend using pixel alpha?!? What am I doing wrong?

I simply need to blend a source pixel onto a destination pixel using the source pixel's alpha channel. I would have thought that this could be accomplished using N2D_BLEND_SRC_OVER as it is described as performing the operation S + (1 - Sa) * D. However, from what I can tell it seems to only use global alpha values for these calculations. I've tried changing the global alpha blend modes for source and destination but it doesn't help. (BTW, what is N2D_GLOBAL_ALPHA_SCALE ????) Any ideas? Thanks!
#6
automate
Senior Member
  • Total Posts : 113
  • Reward points : 0
  • Joined: 2014/11/10 09:58:57
  • Location: 0
  • Status: offline
Re: Legato and Issues with Layer Transparency 2020/10/26 15:46:36 (permalink)
5 (1)
@baugs
 
Use the SetGlobalAlpha API. Set the src global alpha ON and its alpha value for all pixels in the rectangle.  The code snippet below creates CYAN from a blend of BLUE and GREEN src and destination buffers. Change the alpha value for your specific needs.  Let me know if this information helps.
 
   // get the framebuffer
  if (gfxDriverInterface.ioctl(GFX_IOCTL_GET_FRAMEBUFFER, &ioctlArg) == GFX_IOCTL_OK)
   {
       frameBuff = ioctlArg.value.v_pbuffer;
   }
 
  DRV_2DGPU_Fill(&srcBuff, &srcRect, 0xff0000ff);
  DRV_2DGPU_Fill(frameBuff, &destRect, 0x00ff00ff);  // framebuffer  (destination)
  DRV_2DGPU_SetBlend(GFX_BLEND_SRC_OVER);
  DRV_2DGPU_SetGlobalAlpha(GFX_GLOBAL_ALPHA_ON, GFX_GLOBAL_ALPHA_OFF,0x02, 0xff);
  DRV_2DGPU_Blit(&srcBuff, &srcRect, frameBuff, &destRect);
 
 
#7
baugs
New Member
  • Total Posts : 17
  • Reward points : 0
  • Joined: 2009/10/29 13:31:38
  • Location: 0
  • Status: offline
Re: Legato and Issues with Layer Transparency 2020/10/26 17:17:45 (permalink)
4 (1)
Thanks, automate. Unfortunately that does not produce the result I am looking for. I need to blend on a pixel-by-pixel basis where the specific alpha value at each source pixel location is used in the blend calculation of that pixel. Doing it like you described assigns a global alpha to the entire source buffer which overrides the pixel specific alpha values during the blend. Any pixel that was semi-transparent, e.g. pixels used in antialiasing, are essentially converted to solid and thus any antialias effect is lost.
 
#8
automate
Senior Member
  • Total Posts : 113
  • Reward points : 0
  • Joined: 2014/11/10 09:58:57
  • Location: 0
  • Status: offline
Re: Legato and Issues with Layer Transparency 2020/10/27 09:43:02 (permalink)
5 (1)
 
@baugs... "I simply need to blend a source pixel onto a destination pixel using the source pixel's alpha channel."
 
The code snippet below creates a memory region of pixels with varying alpha channels. We call the memory region srcBuff and it is ultimately bounded by srcRect. These pixels could be an image or other surface. The destBuff (target) is bounded by destRect and contains a solid color with opaque alpha 0xff channel.  Here, we use a blue src and green destination to produce various levels of cyan. You can use this to blend over a destination pixels with src pixels. Source pixels with alpha values less than 0xff, will show more of the destination color. Pixels having no color and alpha of 0x00 will be transparent. This method is per-pixel alpha blending.
 
         DRV_2DGPU_Fill(&srcBuff, &srcRect, 0xff0000ff);  //column of blue opaque
                srcRect.x = 50;
                DRV_2DGPU_Fill(&srcBuff, &srcRect, 0xff00007f); // column of blue alpha
                srcRect.x = 100;
                DRV_2DGPU_Fill(&srcBuff, &srcRect, 0xff00003f); // column of blue more transparent
                srcRect.x = 150;
                DRV_2DGPU_Fill(&srcBuff, &srcRect, 0x00000000); //column of transparent pixels
                
               DRV_2DGPU_Fill(frameBuff, &destRect, 0x00ff00ff); // green pixels
                
                DRV_2DGPU_SetBlend(GFX_BLEND_SRC_OVER);
                srcRect.x = 0;
                srcRect.width = 200;
                DRV_2DGPU_Blit(&srcBuff, &srcRect, frameBuff, &destRect);
            

post edited by automate - 2020/10/27 10:20:29
#9
baugs
New Member
  • Total Posts : 17
  • Reward points : 0
  • Joined: 2009/10/29 13:31:38
  • Location: 0
  • Status: offline
Re: Legato and Issues with Layer Transparency 2020/10/28 13:14:08 (permalink)
0
@automate, what you posted makes complete sense and is the approach that I believed would give me the result I wanted. It's also the first thing I had tried. However, it doesn't produce what I am looking for.

I've figured it out though and will explain below. Before I get started, I have to say that I feel a bit silly because I now realize that I had already gone through this whole ordeal over a year ago with Aria and had reached the same conclusions. Apparently, I didn't keep notes on this and having forgot I had to figure it out all over again.

In summary, the issue appears to be a limitation of the available hardware-accelerated blend modes. N2D_BLEND_SRC_OVER blends the colors using the formula S + (1 - Sa) * D. In practice, this mode does indeed do what the formula says. Here is some example RGBA pixel data:

srcBuff                                                                0xFF00_0000    0xFF00_0000    0xFFFF_0010    0xFFFF_0020    0xFF00_0000
frameBuff (before blit):                                         0x0000_00FF    0x0000_00FF    0x0000_00FF    0x0000_00FF    0x0000_00FF
frameBuff (after blit w/N2D_BLEND_SRC_OVER):    0xFF00_00FF    0xFF00_00FF    0xFFFF_00FF    0xFFFF_00FF    0xFF00_00FF
frameBuff (after blit w/N2D_BLEND_NONE):            0xFF00_0000    0xFF00_0000    0xFFFF_0010    0xFFFF_0020    0xFF00_0000
frameBuff (the result I want):                                0x0000_00FF    0x0000_00FF    0x1010_00FF    0x2020_00FF    0x0000_00FF

If you look at the results above, as N2D_BLEND_SRC_OVER is currently implemented, a fully transparent source pixel with a non-zero color will overwrite a black pixel with the source pixel color. What is the utility in this? By the math, it's inline with the formula in the documentation. The issue is that the formula used by N2D_BLEND_SRC_OVER is not what I would expect based on the name and there is no other blend option that will give me the result I need.

N2D_BLEND_SRC_OVER:
actual formula ->                                                             S + (1 - Sa) * D
what I would expect based on the name of the blend ->      S * Sa + (1 - Sa) * D

(I only recently noticed the omission of the Sa factor in the as-implemented formula.)

I am curious to know why the chosen blend formulas were selected for implementation. Apparently the GFX team didn't find them very useful as I have never seen anything used in Aria or Legato other than the default N2D_BLEND_NONE, which is simply a 1:1 blit.

Even more perplexing is that the only blend implemented in software in Legato/Aria is the exact blend that I am asking for! (My understanding is that this is also the most common form of alpha blending.) Currently ALL alpha blending in Legato (and Aria) is done in software on a pixel-by-pixel basis. This is extremely inefficient! Why not use the GPU? In leRenderer, use a second scratch buffer to store the pixel data for the widget being processed. After the widget is processed, use the GPU to alpha blend blit the widget pixels into the primary scratch buffer of the rectangle being processed all in one go. Not only would this be far more processor efficient, it would simplify the code because the software pixel blend routines would no longer be needed. Of course, to do this would require that the GPU library support this type of blend.

Anyways, unless someone knows of a way to do this blend with the GPU, it looks like I'll have to resort back to pixel-by-pixel software blending.
#10
baugs
New Member
  • Total Posts : 17
  • Reward points : 0
  • Joined: 2009/10/29 13:31:38
  • Location: 0
  • Status: offline
Re: Legato and Issues with Layer Transparency 2020/10/28 14:56:14 (permalink)
0
So it occurred to me that I could just do the "S * Sa" part of the calculation over the entire source buffer and then call the blit blend with N2D_BLEND_SRC_OVER. This is much more efficient than doing the full blend in software but certainly slower than the GPU could do it.

size_t counter ;
uint32_t red, green, blue, alpha, pxRes ;
size_t scratchBuffLen = sizeof (_scratchBuffer.data ) / sizeof (uint32_t ) ;
for ( counter = 0 ; counter < scratchBuffLen ; counter++ )
{
   alpha = _scratchBuffer.data[counter] & ALPHA_MASK_32BIT_RGBA ;
   red = ( _scratchBuffer.data[counter] & RED_MASK_32BIT_RGBA ) >> 24 ;
   green = ( _scratchBuffer.data[counter] & GREEN_MASK_32BIT_RGBA ) >> 16 ;
   blue = ( _scratchBuffer.data[counter] & BLUE_MASK_32BIT_RGBA ) >> 8 ;
      
   pxRes = 0 ;
   red *= alpha ;
   green *= alpha ;
   blue *= alpha ;
   pxRes |= ( ( red + 1 + ( red >> 8 ) ) << 16 ) & RED_MASK_32BIT_RGBA ;
   pxRes |= ( ( green + 1 + ( green >> 8 ) ) << 8 ) & GREEN_MASK_32BIT_RGBA ;
   pxRes |= ( blue + 1 + ( blue >> 8 ) ) & BLUE_MASK_32BIT_RGBA ;
   pxRes |= alpha ;
   _scratchBuffer.data[counter] = pxRes ;
}

      
The end result after the blit is the result that I was looking for. It isn't ideal but it works.
#11
Jump to:
© 2020 APG vNext Commercial Version 4.5