CSV format import-export

[Fazekas László]

Hello,

I'm working on a CSV import and export plugin for Krita. CSV is an
animation export format of TVPaint. I already made an import plugin for
Blender as well.

I've no git push rights and almost all of these files are new, so I
attached the files to this mail as a tgz file. Only the
impex/CMakeLists.txt file is modified, all other files are new. To load
or save, use the Open and the Save As items on the File menu, this is
not an animation sequence format.

There are still some questions for me (I'm not familiar with Qt anyway):

- for bigger scenes, it takes a long time to load or save the animation
with this format (the plugin loads or saves a bunch of png files). So I
want to show the status with the progress bar at the bottom of the main
window. What is the correct way to do it? It seems the png importer also
using this same bar, so maybe I have to supress that while showing mine.

- during these long import/export operations, the gui stops working. Is
it a co-operative multitasking thing? What can I do to make it more user
friendly? Is there a callback or something?

Fazek

[Simon Legrand]

Hi Fazekas. I'm not a dev, but simply a user. And I'd like to say that
being able to use Krita in conjunction with TV paint is a great idea. I
look forward to testing your plugin.

Post by Fazekas László
Hello,
I'm working on a CSV import and export plugin for Krita. CSV is an
animation export format of TVPaint. I already made an import plugin for
Blender as well.
I've no git push rights and almost all of these files are new, so I
attached the files to this mail as a tgz file. Only the
impex/CMakeLists.txt file is modified, all other files are new. To load or
save, use the Open and the Save As items on the File menu, this is not an
animation sequence format.
- for bigger scenes, it takes a long time to load or save the animation
with this format (the plugin loads or saves a bunch of png files). So I
want to show the status with the progress bar at the bottom of the main
window. What is the correct way to do it? It seems the png importer also
using this same bar, so maybe I have to supress that while showing mine.
- during these long import/export operations, the gui stops working. Is it
a co-operative multitasking thing? What can I do to make it more user
friendly? Is there a callback or something?
Fazek
_______________________________________________
Krita mailing list
https://mail.kde.org/mailman/listinfo/kimageshop

[Fazekas László]

Post by Simon Legrand
Hi Fazekas. I'm not a dev, but simply a user. And I'd like to say that
being able to use Krita in conjunction with TV paint is a great idea. I
look forward to testing your plugin.

Yes, that's one of the important reasons for me too. TVPaint is only
exports this CSV by itself, (at the "Layers structure" tab in the export
animation window). To import it back into TVPaint, I'm working on a
George script, which I will share on their forum soon.

It seems the Krita loader plugin causes a segfault sometimes when I exit
Krita, so there are bugs yet.

Fazek

Post by Simon Legrand

Post by Fazekas László
Hello,
I'm working on a CSV import and export plugin for Krita. CSV is an
animation export format of TVPaint. I already made an import plugin for
Blender as well.
I've no git push rights and almost all of these files are new, so I
attached the files to this mail as a tgz file. Only the
impex/CMakeLists.txt file is modified, all other files are new. To load or
save, use the Open and the Save As items on the File menu, this is not an
animation sequence format.
- for bigger scenes, it takes a long time to load or save the animation
with this format (the plugin loads or saves a bunch of png files). So I
want to show the status with the progress bar at the bottom of the main
window. What is the correct way to do it? It seems the png importer also
using this same bar, so maybe I have to supress that while showing mine.
- during these long import/export operations, the gui stops working. Is it
a co-operative multitasking thing? What can I do to make it more user
friendly? Is there a callback or something?
Fazek
_______________________________________________
Krita mailing list
https://mail.kde.org/mailman/listinfo/kimageshop

_______________________________________________
Krita mailing list
https://mail.kde.org/mailman/listinfo/kimageshop

[Wolthera]

Please make a diff or task on phabricator.kde.org with the tar added. We're
gonna lose the files if you sent them only through mail, and that'd be sad
:(

Post by Fazekas László
Hello,
I'm working on a CSV import and export plugin for Krita. CSV is an
animation export format of TVPaint. I already made an import plugin for
Blender as well.
I've no git push rights and almost all of these files are new, so I
attached the files to this mail as a tgz file. Only the
impex/CMakeLists.txt file is modified, all other files are new. To load or
save, use the Open and the Save As items on the File menu, this is not an
animation sequence format.
- for bigger scenes, it takes a long time to load or save the animation
with this format (the plugin loads or saves a bunch of png files). So I
want to show the status with the progress bar at the bottom of the main
window. What is the correct way to do it? It seems the png importer also
using this same bar, so maybe I have to supress that while showing mine.
- during these long import/export operations, the gui stops working. Is it
a co-operative multitasking thing? What can I do to make it more user
friendly? Is there a callback or something?
Fazek
_______________________________________________
Krita mailing list
https://mail.kde.org/mailman/listinfo/kimageshop

[Boudewijn Rempt]

Post by Fazekas László
Hello,
I'm working on a CSV import and export plugin for Krita. CSV is an animation
export format of TVPaint. I already made an import plugin for Blender as
well.
I've no git push rights and almost all of these files are new,

If you've gotten this far, it might be a good idea to get you git push rights!
It's not that hard, once you've got a KDE identity (needed for phabricator),
you only need a sponsor to get push access. And then you can make your development
branch and it would be really easy for everyone to follow your work!

Post by Fazekas László
so I attached
the files to this mail as a tgz file. Only the impex/CMakeLists.txt file is
modified, all other files are new. To load or save, use the Open and the Save
As items on the File menu, this is not an animation sequence format.
- for bigger scenes, it takes a long time to load or save the animation with
this format (the plugin loads or saves a bunch of png files). So I want to
show the status with the progress bar at the bottom of the main window. What
is the correct way to do it? It seems the png importer also using this same
bar, so maybe I have to supress that while showing mine.
- during these long import/export operations, the gui stops working. Is it a
co-operative multitasking thing? What can I do to make it more user friendly?
Is there a callback or something?
Fazek

[Fazekas László]

Hello Boud,

I hope I can do it very soon. But for now, I created a diff on
Phabricator (D1068) and added you (rempt) also as a reviewer if you
don't mind. Also thanks for the advice about the png part. I changed the
png export, if you have free time, please take a look at it, it's in the
csv_saver.cpp file from line 385.

Fazek

Post by Boudewijn Rempt

Post by Fazekas László
Hello,
I'm working on a CSV import and export plugin for Krita. CSV is an
animation export format of TVPaint. I already made an import plugin
for Blender as well.
I've no git push rights and almost all of these files are new,

If you've gotten this far, it might be a good idea to get you git push
rights! It's not that hard, once you've got a KDE identity (needed for
phabricator),
you only need a sponsor to get push access. And then you can make your development
branch and it would be really easy for everyone to follow your work!

Post by Fazekas László
so I attached the files to this mail as a tgz file. Only the
impex/CMakeLists.txt file is modified, all other files are new. To
load or save, use the Open and the Save As items on the File menu,
this is not an animation sequence format.
- for bigger scenes, it takes a long time to load or save the
animation with this format (the plugin loads or saves a bunch of png
files). So I want to show the status with the progress bar at the
bottom of the main window. What is the correct way to do it? It seems
the png importer also using this same bar, so maybe I have to supress
that while showing mine.
- during these long import/export operations, the gui stops working.
Is it a co-operative multitasking thing? What can I do to make it
more user friendly? Is there a callback or something?
Fazek

_______________________________________________
Krita mailing list
https://mail.kde.org/mailman/listinfo/kimageshop

[Boudewijn Rempt]

Hi Fazek,

Yes, that looks correct! I'm right now testing another patch, so I haven't built it yet, but your plugin looks really good to me!

Boudewijn

Post by Fazekas László
Hello Boud,
I hope I can do it very soon. But for now, I created a diff on Phabricator (D1068) and added you (rempt) also as a reviewer if you don't mind. Also thanks for the advice about the png
part. I changed the png export, if you have free time, please take a look at it, it's in the csv_saver.cpp file from line 385.
Fazek
Hello,
I'm working on a CSV import and export plugin for Krita. CSV is an animation export format of TVPaint. I already made an import plugin for Blender as well.
I've no git push rights and almost all of these files are new,
If you've gotten this far, it might be a good idea to get you git push rights! It's not that hard, once you've got a KDE identity (needed for phabricator),
you only need a sponsor to get push access. And then you can make your development
branch and it would be really easy for everyone to follow your work!
so I attached the files to this mail as a tgz file. Only the impex/CMakeLists.txt file is modified, all other files are new. To load or save, use the Open and
the Save As items on the File menu, this is not an animation sequence format.
- for bigger scenes, it takes a long time to load or save the animation with this format (the plugin loads or saves a bunch of png files). So I want to show the
status with the progress bar at the bottom of the main window. What is the correct way to do it? It seems the png importer also using this same bar, so maybe I
have to supress that while showing mine.
- during these long import/export operations, the gui stops working. Is it a co-operative multitasking thing? What can I do to make it more user friendly? Is
there a callback or something?
Fazek
_______________________________________________
Krita mailing list
https://mail.kde.org/mailman/listinfo/kimageshop

[Dmitry Kazakov]

Hi, Laszlo!

- for bigger scenes, it takes a long time to load or save the animation

Post by Fazekas László
with this format (the plugin loads or saves a bunch of png files). So I
want to show the status with the progress bar at the bottom of the main
window. What is the correct way to do it? It seems the png importer also
using this same bar, so maybe I have to supress that while showing mine.
- during these long import/export operations, the gui stops working. Is it
a co-operative multitasking thing? What can I do to make it more user
friendly? Is there a callback or something?

As far as I know, in the import/export plugins we don't have any progress
reporting framework (Boud will correct me if I'm wrong), though it might be
a good idea to have one. I guess you can try to implement one. The entry
point to access a progress bar in the bottom is:
KisViewManager::createProgressUpdater(). The saving itself happens in
KisDocument. It uses d->filterManager->exportDocument() to save the
documents. You can add some signal to the export filter and bind it to the
document's signal. The KisViewManager can connect to this signal on
switching views and show the progress to the user.

Actually because of this reason in "Export Animation" action we use a
"hackish" approach to show a separate dialog with a progress bar. We just
don't use export plugins at all, but this is wrong, of course.

[Boudewijn Rempt]

Post by Dmitry Kazakov
Hi, Laszlo!
- for bigger scenes, it takes a long time to load or save the animation with this format (the plugin loads or saves a bunch of png files). So I want to show the status with
the progress bar at the bottom of the main window. What is the correct way to do it? It seems the png importer also using this same bar, so maybe I have to supress that
while showing mine.
- during these long import/export operations, the gui stops working. Is it a co-operative multitasking thing? What can I do to make it more user friendly? Is there a
callback or something?
As far as I know, in the import/export plugins we don't have any progress reporting framework (Boud will correct me if I'm wrong), though it might be a good idea to have one.

There actually is one, you can set an updater using KisImportExportFilter::setUpdater and report progress with the sigProgress(int percent); whether that actually works is another thing. Adding progress reporting to our import/export plugins has been a todo since we moved away from the built-in ImageMagick filters, I think!

But the proper thing to do is emit that signal from the filter and then let me worry about how whether the progress bar is shown.

Post by Dmitry Kazakov
I guess
you can try to implement one. The entry point to access a progress bar in the bottom is: KisViewManager::createProgressUpdater(). The saving itself happens in KisDocument. It uses
d->filterManager->exportDocument() to save the documents. You can add some signal to the export filter and bind it to the document's signal. The KisViewManager can connect to this
signal on switching views and show the progress to the user.
Actually because of this reason in "Export Animation" action we use a "hackish" approach to show a separate dialog with a progress bar. We just don't use export plugins at all, but this
is wrong, of course.
--
Dmitry Kazakov

[Boudewijn Rempt]

I checked, and the updater is set, the progressbar is created, so it's really
only a matter of emitting the signal. Calculating the percentage is likely to
be complicated, though...

Post by Fazekas László

Post by Dmitry Kazakov
Hi, Laszlo!
- for bigger scenes, it takes a long time to load or save the

animation with this format (the plugin loads or saves a bunch of png files).
So I want to show the status with

Post by Dmitry Kazakov
the progress bar at the bottom of the main window. What is the

correct way to do it? It seems the png importer also using this same bar, so
maybe I have to supress that

Post by Dmitry Kazakov
while showing mine.
- during these long import/export operations, the gui stops working.

Is it a co-operative multitasking thing? What can I do to make it more user
friendly? Is there a

Post by Dmitry Kazakov
callback or something?
As far as I know, in the import/export plugins we don't have any progress

reporting framework (Boud will correct me if I'm wrong), though it might be a
good idea to have one.
There actually is one, you can set an updater using
KisImportExportFilter::setUpdater and report progress with the
sigProgress(int percent); whether that actually works is another thing.
Adding progress reporting to our import/export plugins has been a todo since
we moved away from the built-in ImageMagick filters, I think!
But the proper thing to do is emit that signal from the filter and then let
me worry about how whether the progress bar is shown.

Post by Dmitry Kazakov
I guess
you can try to implement one. The entry point to access a progress bar in

the bottom is: KisViewManager::createProgressUpdater(). The saving itself
happens in KisDocument. It uses

Post by Dmitry Kazakov
d->filterManager->exportDocument() to save the documents. You can add some

signal to the export filter and bind it to the document's signal. The
KisViewManager can connect to this

Post by Dmitry Kazakov
signal on switching views and show the progress to the user.
Actually because of this reason in "Export Animation" action we use a

"hackish" approach to show a separate dialog with a progress bar. We just
don't use export plugins at all, but this

Post by Dmitry Kazakov
is wrong, of course.
--
Dmitry Kazakov

[Dmitry Kazakov]

Post by Dmitry Kazakov
As far as I know, in the import/export plugins we don't have any progress

Post by Dmitry Kazakov
reporting framework (Boud will correct me if I'm wrong), though it might be
a good idea to have one.

There actually is one, you can set an updater using
KisImportExportFilter::setUpdater and report progress with the
sigProgress(int percent); whether that actually works is another thing.
Adding progress reporting to our import/export plugins has been a todo
since we moved away from the built-in ImageMagick filters, I think!
But the proper thing to do is emit that signal from the filter and then
let me worry about how whether the progress bar is shown.

Yes, that would be an ideal solution :)

[Fazekas László]

Sorry for the late answer, I was away from the computer.

There must be something already with the progressbar, because when I
call the image import for the png files, the progressbar at the bottom
flashes. Only it always shows 0%.

I don't know how it works now, I'm going to check the code tomorrow. We
should be able to handle the race condition if one function using the
progressbar calls another, and the called part also wants to set one for
itself.

Let's say there is function Alice() and she sends progressbar updates
while working. During this, it calls function Bob() and he also reports
its readiness. It's wrong to send all these values directly to the
widget, the progressbar would flash like crazy. The good value for the
widget is lerp( current(Alice), next(Alice), current(Bob) ). But the
progressbar doesn't know the next(Alice) value - probably even Alice()
can't calculate it before it really happens. And there can be a third
Carole() function called by Bob() too...

One solution could be to somehow separate the signals by their source
and keep all these values in a list. I think the percentage report
should contain an optional next value for the correct calculations.
Without the provided next value, the lower levels should be ignored.

But perhaps this is too complicated so it's easier to entirely disable
the embedded reporting. In this case the initial setUpdater() call for
Bob() should return an error code and tell Bob() that he must be silent.

Fazek

Post by Dmitry Kazakov

Post by Dmitry Kazakov
As far as I know, in the import/export plugins we don't have any progress

Post by Dmitry Kazakov
reporting framework (Boud will correct me if I'm wrong), though it might be
a good idea to have one.

There actually is one, you can set an updater using
KisImportExportFilter::setUpdater and report progress with the
sigProgress(int percent); whether that actually works is another thing.
Adding progress reporting to our import/export plugins has been a todo
since we moved away from the built-in ImageMagick filters, I think!
But the proper thing to do is emit that signal from the filter and then
let me worry about how whether the progress bar is shown.

Yes, that would be an ideal solution :)
_______________________________________________
Krita mailing list
https://mail.kde.org/mailman/listinfo/kimageshop

[Boudewijn Rempt]

Post by Fazekas László
Sorry for the late answer, I was away from the computer.
There must be something already with the progressbar, because when I call the image import for the png files, the progressbar at the bottom flashes. Only
it always shows 0%.

Yeah. I started digging into that today... And somehow, over the past ten years, it got broken in weird and wonderful ways! I know what's up, though, and I'll try to fix it tomorrow, but I've got a funeral to serve at first... But as long as you can emit a signal with percentage progress, I'll make sure the user is informed of it :-)

Post by Fazekas László
I don't know how it works now, I'm going to check the code tomorrow. We should be able to handle the race condition if one function using the progressbar
calls another, and the called part also wants to set one for itself.
Let's say there is function Alice() and she sends progressbar updates while working. During this, it calls function Bob() and he also reports its
readiness. It's wrong to send all these values directly to the widget, the progressbar would flash like crazy. The good value for the widget is lerp(
current(Alice), next(Alice), current(Bob) ). But the progressbar doesn't know the next(Alice) value - probably even Alice() can't calculate it before it
really happens. And there can be a third Carole() function called by Bob() too...
One solution could be to somehow separate the signals by their source and keep all these values in a list. I think the percentage report should contain an
optional next value for the correct calculations. Without the provided next value, the lower levels should be ignored.
But perhaps this is too complicated so it's easier to entirely disable the embedded reporting. In this case the initial setUpdater() call for Bob() should
return an error code and tell Bob() that he must be silent.
Fazek
As far as I know, in the import/export plugins we don't have any progress
reporting framework (Boud will correct me if I'm wrong), though it might be
a good idea to have one.
There actually is one, you can set an updater using
KisImportExportFilter::setUpdater and report progress with the
sigProgress(int percent); whether that actually works is another thing.
Adding progress reporting to our import/export plugins has been a todo
since we moved away from the built-in ImageMagick filters, I think!
But the proper thing to do is emit that signal from the filter and then
let me worry about how whether the progress bar is shown.
Yes, that would be an ideal solution :)
_______________________________________________
Krita mailing list
https://mail.kde.org/mailman/listinfo/kimageshop