Interactive Visualization & Near Real-time Analysis | Draga Doncila Pop | Dask Summit 2021
1.2 هزار بار بازدید -
3 سال پیش
-
This talk demonstrates interactive visualization
This talk demonstrates interactive visualization & near real-time analysis and was given by Draga Doncila Pop at Dask Summit 2021.
We demonstrate a practical case study: loading and interacting with hundreds of Sentinel satellite images and the results of their analysis in near real-time. The project was part of research on interactive visualization of out-of-core images, in the context of the Monash VegMap land cover study. We used dask and napari to reduce the required RAM from 150 GB to something that could be run on a laptop.
Improvements in imaging technology have led to high resolution satellites, microscopes, scanners and telescopes acquiring single images upwards of 100GB in volume - far beyond the RAM capability of a typical computer core. The visualization tools provided by current software in python are in their infancy, with little support for high dimensional images and results visualization. This leads to a fragmented workflow for users, and a high bar to entry for extending or customizing functionality.
Human inspection of these images forms a key part of the research process. Frequent visualization in between processing and analysis steps, as well as during the exploratory phase, is critical for effective analysis and ensuring the correctness of results.
This talk introduces napari- a fast, general purpose image viewer in python- in the context of a remote-sensing application on Sentinel satellite images. It is targeted both at existing python programmers who wish to view complex imaging datasets in real time and develop their own functionality on top of the viewer, and those new to python who want to make the most out of their data without needing to write hundreds of lines of code. In more detail this talk will cover:
- An overview of napari
- An introduction to extending napari through its plugin ecosystem
- A demonstration of a system supporting human-in-the-loop analysis of Sentinel satellite images and their associated result data
This talk will introduce new tools to support imaging research workflows and give confidence to both new and experienced python programmers that effective, performant, near-real time out-of-core visualization is possible even when dealing with vast imaging datasets. All software and packages discussed throughout this talk are open-source and available at the following links:
napari : https://napari.org/docs/dev/
napari plugin guide: https://napari.org/docs/dev/plugins/f...
napari-sentinel-to-zarr: https://github.com/DragaDoncila/napar...
What is the Dask Summit?
The Dask Distributed Summit is where users, contributors, and newcomers can share experiences to learn from one another and grow together. The Dask Distributed Summit provides content, information, and learning opportunities for attendees of all levels of Dask familiarity and expertise.
Share your feedback with us in the comments and let us know:
- Did you find this talk on interactive visualization helpful?
- What is your experience with interactive visualization?
Learn more at summit.dask.org and dask.org
KEY MOMENTS
00:00:00:00 Intro
00:01:20:11 Big Data, Low Effort
00:01:49:20 Napari Overview
00:03:36:21 Different Layer Types
00:05:42:16 Napari is Lazy By Nature
00:06:39:21 Extensible with Plugins
00:08:33:16 Looking at Sentinel 2A Satellite Images
00:11:38:20 Raw Data Reader
00:13:52:21 Interpolated Data
00:15:46:20 Labels
00:16:42:11 Results!
00:22:30:13 Limitations
00:24:11:10 Conclusions
00:24:54:23 Dask is Indispensable
We demonstrate a practical case study: loading and interacting with hundreds of Sentinel satellite images and the results of their analysis in near real-time. The project was part of research on interactive visualization of out-of-core images, in the context of the Monash VegMap land cover study. We used dask and napari to reduce the required RAM from 150 GB to something that could be run on a laptop.
Improvements in imaging technology have led to high resolution satellites, microscopes, scanners and telescopes acquiring single images upwards of 100GB in volume - far beyond the RAM capability of a typical computer core. The visualization tools provided by current software in python are in their infancy, with little support for high dimensional images and results visualization. This leads to a fragmented workflow for users, and a high bar to entry for extending or customizing functionality.
Human inspection of these images forms a key part of the research process. Frequent visualization in between processing and analysis steps, as well as during the exploratory phase, is critical for effective analysis and ensuring the correctness of results.
This talk introduces napari- a fast, general purpose image viewer in python- in the context of a remote-sensing application on Sentinel satellite images. It is targeted both at existing python programmers who wish to view complex imaging datasets in real time and develop their own functionality on top of the viewer, and those new to python who want to make the most out of their data without needing to write hundreds of lines of code. In more detail this talk will cover:
- An overview of napari
- An introduction to extending napari through its plugin ecosystem
- A demonstration of a system supporting human-in-the-loop analysis of Sentinel satellite images and their associated result data
This talk will introduce new tools to support imaging research workflows and give confidence to both new and experienced python programmers that effective, performant, near-real time out-of-core visualization is possible even when dealing with vast imaging datasets. All software and packages discussed throughout this talk are open-source and available at the following links:
napari : https://napari.org/docs/dev/
napari plugin guide: https://napari.org/docs/dev/plugins/f...
napari-sentinel-to-zarr: https://github.com/DragaDoncila/napar...
What is the Dask Summit?
The Dask Distributed Summit is where users, contributors, and newcomers can share experiences to learn from one another and grow together. The Dask Distributed Summit provides content, information, and learning opportunities for attendees of all levels of Dask familiarity and expertise.
Share your feedback with us in the comments and let us know:
- Did you find this talk on interactive visualization helpful?
- What is your experience with interactive visualization?
Learn more at summit.dask.org and dask.org
KEY MOMENTS
00:00:00:00 Intro
00:01:20:11 Big Data, Low Effort
00:01:49:20 Napari Overview
00:03:36:21 Different Layer Types
00:05:42:16 Napari is Lazy By Nature
00:06:39:21 Extensible with Plugins
00:08:33:16 Looking at Sentinel 2A Satellite Images
00:11:38:20 Raw Data Reader
00:13:52:21 Interpolated Data
00:15:46:20 Labels
00:16:42:11 Results!
00:22:30:13 Limitations
00:24:11:10 Conclusions
00:24:54:23 Dask is Indispensable
3 سال پیش
در تاریخ 1400/04/01 منتشر شده
است.
1,268
بـار بازدید شده