Fight for Synedit speed under qt widgetset

One or two weeks ago our qt-ide was almost fine (if form designer was not taken into account) except one important detail - the speed of the unit editor. Yes, you were able to edit source, but the speed .... it was deadly slow. You could type a key and wait second or two before that key was printed in editor. So we started the fight for speed.

The core of the unit editor is the Synedit component. Lazarus uses an early version of Synedit but it is heavily changed for our needs (adopted, extended). Synedit is a TCustomControl, so it does all its painting and event handling itself. When I looked at the source for the first time, I found nothing criminal. A well done component which repaints only changed areas.

I tried to find out what piece of code makes the component so slow under qt. And I found that it is the qt QPainter.DrawText method. My first assumption was that in the OnPaint event we were repainting the whole Synedit, not only the changed area.

I checked how invalidateRect was done and there was no error - QWidget.Update(Rect) was called. My next assumption was that we were ignoring that Rect in the OnPaint event, but no - we did clip the paint area by that rect. What was it - a mystery?

I checked how clipping worked and found, that in DrawText the clipping was gone. The reason was in Synedit - it used double buffered painting and as result DrawText performed on a non clipped bitmap instead of a clipped window. Revision 12175 changed things. There was no more bitmap, but painting was still as slow as before.

My next assumption was that qt doesn't check the clipping region before DrawText. And bingo - manual check of clipping rect before DrawText gave us no bad speed (revision 12183). But cpu usage was very big in spite of the achieved speed. So something was still wrong, but what?

Zeljan noticed that caret eats too much cpu time. It was because of a not optimal use of timer and widget repainting. After fixing those things in revisions 12198-12200 there was not so high cpu rate as before.

I was already happy but Zeljan was at his best and found one more speed killer. That was our ExtTextOut implementation. ExtTextOut have 2 arguments related to text: Str - the text and Count - count of chars. We made the assumption that Pascal components pass Str arguments that are already trimmed to count chars, but this was not true (at least for Synedit). There were cases where count was equal to 1, but Length of Str equal to 300 (or 400 chars). After an appropriate patch (revisions 12201,12205)the speed of Synedit has become really fast.

Now qt-ide as fast as it should be.

The Lazarus 0.9.24 release date

One of the best held secrets in the Lazarus community is the release date of Lazarus 0.9.24. Sometimes people ask when the next Lazarus release will be. The only correct answer is: "When it is finished". And even the Lazarus developers don't know when that is.

Fortunately, we have a criterium for finished. We want our next release as good as the previous one, so things that work in 0.9.22, must work in 0.9.24 too. If soemthing stops working and it is found out by people using the snapshots, we ask create a bug report for it in the Lazarus bug tracker. Such regressions are marked with "LazTarget 0.9.24". On the view issues page, you can filter on those issues. The list of 0.9.24 issues is currently 23 long, most them are already assigned to some developer. The list also contains patches contributed by Lazarus users, we want to include in the next release. Currently the list is too long to give a reliable estimate of the next Lazarus release date.

In the final stages before a release and during the release preparation we use the Detailed todo page on the wiki as a kind of check list.

You might wonder if nothing is happening on the Lazarus front. The truth is, that very much has been done since 0.9.22. Almost 200 bugs have been fixed (see Changelog in the bug tracker) in Lazarus 0.9.23 and more than 1300 svn commits have been made. You can track the svn commits to the Lazarus svn repository though rss or at the Lazarus page at CIA.vc.

Lazarus snapshot are using fpc 2.2.0

In the last couple of days I have made changes to the build servers and the mirrors. All snapshots that were using fpc 2.0.4 are now using fpc 2.2.0.

The next Lazarus release (version 0.9.24) will be based on fpc 2.2.0 too, so these snapshots give a good preview of the next release. At his moment there are still about 20 bugs open that need to be fixed before the release, so there is still plenty time for testing Lazarus 0.9.23. Download it from one of the snapshot mirrors:

• Hungary
• Germany
  
• USA
  

Reviewing a patch for the chart component

The chart component in Lazarus was written two years ago by Philippe Martinole for his TeleAuto project. He submitted the component to the Lazarus Code and Component Repository. Later Luis Rodrigues extended it and made it more Delphi compatible. The component was considered stable and useful enough to add it to the default installation of Lazarus. Luis Rodrigues now is the main maintainer of the component.

A couple of weeks ago Luis submitted a patch for the chart component. Because I don't know the inner workings of the chart components and don't have a test program, I usually do two things:

• Do a visual inspection of the patch to see if it contains sensible things.
  
• Check if the patched version compiles on under windows, my main development OS.

People submitting these patch are major users of the component, so I assume they are better testers than I possibly can be.

This particular patch contained the following code to fix clipping:

      //set cliping region so we don't draw outsite
-     Rgn := CreateRectRgn(XImageMin, YImageMax, XImageMax, YImageMin);
+     p[0].x := XImageMin;
+     p[0].y := YImageMax;
+     p[1].x := XImageMax;
+     p[1].y := YImageMin;
+
+     {$IFDEF windows}
+     LPtoDP(Canvas.Handle, p, 2);
+     {$ENDIF}
+     Rgn := CreateRectRgn(p[0].x, p[0].y, p[1].x, p[1].y);
    SelectClipRgn (Canvas.Handle, Rgn);

I don't know how clipping works and what LPtoDP exactly does, but having a {$IFDEF} here in the middle of a procedure, doesn't look like good portable code. There seemed to be some inconstencies between the CreateRectRgn functions for win32 widgetset and the gtk widgetset: one expects logical points and the other device points.

Even if we wanted to use a {$IFDEF} the current one is probably not correct. The difference is not the OS, but the used widget set, so using {$IFDEF LCLwin32} would have been better. Remember, that the qt and the gtk2 widget can be compiled on windows too.

I asked Luis to find a better solution and maybe some people on the mailing list could help (unfortunately I had no clue how to fix this properly). Luis found another way without IFDEFS:

      //set cliping region so we don't draw outsite
-     Rgn := CreateRectRgn(XImageMin, YImageMax, XImageMax, YImageMin);
-     SelectClipRgn (Canvas.Handle, Rgn);
+     IntersectClipRect(Canvas.Handle, XImageMin, YImageMax, XImageMax, YImageMin);

This patch I happily applied after I tested the compilation.

Vanishing snapshots

From time to time, the Lazarus snapshots at Scenergy disapear. It is probably happens when two build servers upload a new version at the same time. The update process has roughly the following steps:

1. Delete the files currently in the temp location
   
2. Copy current files from the ftp-directory to a temp location.
3. Remove old snapshot
4. Add new snapshot
5. Rsync the temp location to the ftp-directory

The temp location is the same directory for each upload. Now if one build server is at step 4 and a second build server starts at step 1, the rsync job will finish quickly with no files to copy ...

For normal snapshots this is not a big problem, they will be regenerated within 24 hours. But occasionally I upload a wince cross-installer too and that doesn't get updated automatically yet. So until this issue has been fixed (probably by adding some locking and using different temp locations), I have put the latest wince snapshot at sourceforge, in the Lazarus Testing package.

Some stats

Two weeks ago I setup an ubuntu deb repository for Lazarus releases. I wondered how much they were used and asked the server admin to install webalizer on the server (it's name is Scenergy). In these two weeks, the repository generated 15 GB trafic and the i386 archicture is still favorite.

Scenergy is also one of the Lazarus snapshot servers. So I added the ftp stats too. In August this server served 50 GB of snapshots. The same ftp server has fpc releases too and the fpc 2.0.4 windows installer causes 43% of trafic: 128 GB. Interesting figures ...