{"id":20945,"date":"2022-11-12T09:03:34","date_gmt":"2022-11-12T09:03:34","guid":{"rendered":"https:\/\/vr-room.ch\/?p=20945"},"modified":"2022-11-07T09:14:29","modified_gmt":"2022-11-07T09:14:29","slug":"nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr","status":"publish","type":"post","link":"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/","title":{"rendered":"NeRF player streams dynamic volumetric scenes - also for VR"},"content":{"rendered":"<h3 id=\"nerfs-repraesentieren-3d-szenen-in-einem-neuronalen-netz-eine-neue-arbeit-eines-internationalen-teams-zu-dem-auch-die-eth-zuerich-gehoert-oeffnet-die-ki-technologie-fuer-dynamische-szenen\">NeRFs represent 3D scenes in a neural network. A new paper by an international team including ETH Zurich opens up AI technology to dynamic scenes.<\/h3>\n<p>In summer 2020, Google researchers presented <a href=\"https:\/\/the-decoder.de\/die-welt-digital-in-3d-googles-neue-ki-versteht-das-licht\">Neural Radiance Fields <\/a>(NERF) is planning to launch a <a href=\"https:\/\/the-decoder.de\/kuenstliche-intelligenz-begriffe-erklaerung\">AI<\/a>-process that can extract 3D depth data from 2D images such as photos. NERF can thus create a textured 3D model from multiple photos shot from different angles. Neural Radiance Fields (NeRFs) can thus render 3D scenes from previously unseen angles. This enables, for example, a 360-degree camera tour around an object, a tour via drone shot or a flight through the interior of a restaurant. The technology can thus be used to generate photo-realistic 3D objects.<\/p>\n<p>In almost all cases, however, these are static scenes or objects, since movements introduce a temporal dimension into the training process that has been difficult to resolve so far.<\/p>\n<h3 id=\"nerfs-fuer-dynamische-szenen\">NeRFs for dynamic scenes<\/h3>\n<p>In a new research paper, a team from the University at Buffalo, ETH Zurich, InnoPeak Technology and the University of T\u00fcbingen now shows how NeRFs can represent dynamic scenes and thus learn a 4D representation.<\/p>\n<p>RGB images from different cameras or a single moving camera serve as input. The images show people moving or someone pouring coffee into a glass.<\/p>\n<div id=\"attachment_9655\" class=\"wp-caption alignnone\"><a href=\"https:\/\/the-decoder.de\/wp-content\/uploads\/2022\/11\/NeRFPlayer-decomposition.jpg\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-9655 size-medium entered litespeed-loaded\" src=\"https:\/\/the-decoder.de\/wp-content\/uploads\/2022\/11\/NeRFPlayer-decomposition-770x582.jpg\" alt=\"\" width=\"770\" height=\"582\" data-lazyloaded=\"1\" aria-describedby=\"caption-attachment-9655\" data-src=\"https:\/\/the-decoder.de\/wp-content\/uploads\/2022\/11\/NeRFPlayer-decomposition-770x582.jpg\" data-ll-status=\"loaded\" \/><\/a>For NeRFs to learn dynamic scenes, a so-called decomposition field divides the scene into different areas and then the team divides dynamic scenes into three temporal patterns: static, deforming and new areas.<\/p>\n<\/div>\n<p>In the case of the coffee, for example, the wooden board on which the glass stands remains static. The contents of the glass appear anew and the visible hand deforms. A decomposition field takes over the division of the scene into the three categories. Each area is represented by its own neural field. In their approach, the researchers also decouple temporal and spatial dimensions to improve the representation.<\/p>\n<h3 id=\"virtual-reality-als-vision\">\u00a0Virtual reality as a vision<\/h3>\n<p>The decompositional representation of the dynamic scene significantly reduces visual artefacts compared to other approaches. The team also demonstrates with NeRFPlayer a way to stream the learned representations in real time with limited bit rates.<\/p>\n<p>Through Nvidia's framework <a href=\"https:\/\/the-decoder.de\/nvidia-instant-ngp-ki-grafik-ist-weiter-auf-dem-vormarsch\/\" target=\"_blank\" rel=\"noopener\">InstantNGP<\/a>, with which a neural network can learn representations of gigapixel images, 3D objects and NeRFs within seconds, the method presented is also fast.<\/p>\n<p>In the work, the team describes the visual exploration of a real 4D spatio-temporal environment for virtual reality as a vision and sees their own work as a contribution to this goal.<\/p>\n<p>\"Visual exploration of a real 4D space freely in VR is a long-standing task. The task is particularly appealing when only a few or even a single RGB camera is used to capture the dynamic scene,\" the paper states.<\/p>\n<div class=\"video-block responsive-container\"><\/div>\n<p>The NeRF demonstrations have their origins in a Google research paper from 2020 on spatially filmed light field videos.<\/p>\n<div id=\"content-end\"><\/div>\n<div class=\"article-source\">Source: Mixed \/ the decoder<\/div>","protected":false},"excerpt":{"rendered":"NeRFs represent 3D scenes in a neural network. A new paper by an international team, including ETH...","protected":false},"author":12,"featured_media":20946,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[68],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v22.3 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>NeRF-Player streamt dynamische volumetrische Szenen \u2013 auch f\u00fcr VR - Schweizer Virtual Reality News<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"NeRF-Player streamt dynamische volumetrische Szenen \u2013 auch f\u00fcr VR - Schweizer Virtual Reality News\" \/>\n<meta property=\"og:description\" content=\"NeRFs repr\u00e4sentieren 3D-Szenen in einem neuronalen Netz. Eine neue Arbeit eines internationalen Teams, zu dem auch die ETH&hellip;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/\" \/>\n<meta property=\"og:site_name\" content=\"Schweizer Virtual Reality News\" \/>\n<meta property=\"article:published_time\" content=\"2022-11-12T09:03:34+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-11-07T09:14:29+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/vr-room.ch\/wp-content\/uploads\/2022\/11\/NeRFPlayer_-A-Streamable-Dynamic-Scene-Representation-with-Decomposed-Neural-Radiance-Fields-3-52-screenshot-e1667664753387-1200x754-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"754\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Nathalie\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Nathalie\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/\",\"url\":\"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/\",\"name\":\"NeRF-Player streamt dynamische volumetrische Szenen \u2013 auch f\u00fcr VR - Schweizer Virtual Reality News\",\"isPartOf\":{\"@id\":\"http:\/\/vr-room.ch\/en\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/vr-room.ch\/wp-content\/uploads\/2022\/11\/NeRFPlayer_-A-Streamable-Dynamic-Scene-Representation-with-Decomposed-Neural-Radiance-Fields-3-52-screenshot-e1667664753387-1200x754-1.png\",\"datePublished\":\"2022-11-12T09:03:34+00:00\",\"dateModified\":\"2022-11-07T09:14:29+00:00\",\"author\":{\"@id\":\"http:\/\/vr-room.ch\/en\/#\/schema\/person\/1a03b3a6ced429fb4831f60fa2b37599\"},\"breadcrumb\":{\"@id\":\"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/#primaryimage\",\"url\":\"https:\/\/vr-room.ch\/wp-content\/uploads\/2022\/11\/NeRFPlayer_-A-Streamable-Dynamic-Scene-Representation-with-Decomposed-Neural-Radiance-Fields-3-52-screenshot-e1667664753387-1200x754-1.png\",\"contentUrl\":\"https:\/\/vr-room.ch\/wp-content\/uploads\/2022\/11\/NeRFPlayer_-A-Streamable-Dynamic-Scene-Representation-with-Decomposed-Neural-Radiance-Fields-3-52-screenshot-e1667664753387-1200x754-1.png\",\"width\":1200,\"height\":754,\"caption\":\"Nerf\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Startseite\",\"item\":\"http:\/\/vr-room.ch\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"NeRF-Player streamt dynamische volumetrische Szenen \u2013 auch f\u00fcr VR\"}]},{\"@type\":\"WebSite\",\"@id\":\"http:\/\/vr-room.ch\/en\/#website\",\"url\":\"http:\/\/vr-room.ch\/en\/\",\"name\":\"Schweizer Virtual Reality News\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"http:\/\/vr-room.ch\/en\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Person\",\"@id\":\"http:\/\/vr-room.ch\/en\/#\/schema\/person\/1a03b3a6ced429fb4831f60fa2b37599\",\"name\":\"Nathalie\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"http:\/\/vr-room.ch\/en\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/vr-room.ch\/wp-content\/uploads\/2017\/01\/avatar_user_12_1484159890-96x96.jpg\",\"contentUrl\":\"https:\/\/vr-room.ch\/wp-content\/uploads\/2017\/01\/avatar_user_12_1484159890-96x96.jpg\",\"caption\":\"Nathalie\"},\"url\":\"https:\/\/vr-room.ch\/en\/author\/nathalie\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"NeRF-Player streamt dynamische volumetrische Szenen \u2013 auch f\u00fcr VR - Schweizer Virtual Reality News","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/","og_locale":"en_GB","og_type":"article","og_title":"NeRF-Player streamt dynamische volumetrische Szenen \u2013 auch f\u00fcr VR - Schweizer Virtual Reality News","og_description":"NeRFs repr\u00e4sentieren 3D-Szenen in einem neuronalen Netz. Eine neue Arbeit eines internationalen Teams, zu dem auch die ETH&hellip;","og_url":"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/","og_site_name":"Schweizer Virtual Reality News","article_published_time":"2022-11-12T09:03:34+00:00","article_modified_time":"2022-11-07T09:14:29+00:00","og_image":[{"width":1200,"height":754,"url":"https:\/\/vr-room.ch\/wp-content\/uploads\/2022\/11\/NeRFPlayer_-A-Streamable-Dynamic-Scene-Representation-with-Decomposed-Neural-Radiance-Fields-3-52-screenshot-e1667664753387-1200x754-1.png","type":"image\/png"}],"author":"Nathalie","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Nathalie","Estimated reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/","url":"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/","name":"NeRF-Player streamt dynamische volumetrische Szenen \u2013 auch f\u00fcr VR - Schweizer Virtual Reality News","isPartOf":{"@id":"http:\/\/vr-room.ch\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/#primaryimage"},"image":{"@id":"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/#primaryimage"},"thumbnailUrl":"https:\/\/vr-room.ch\/wp-content\/uploads\/2022\/11\/NeRFPlayer_-A-Streamable-Dynamic-Scene-Representation-with-Decomposed-Neural-Radiance-Fields-3-52-screenshot-e1667664753387-1200x754-1.png","datePublished":"2022-11-12T09:03:34+00:00","dateModified":"2022-11-07T09:14:29+00:00","author":{"@id":"http:\/\/vr-room.ch\/en\/#\/schema\/person\/1a03b3a6ced429fb4831f60fa2b37599"},"breadcrumb":{"@id":"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/"]}]},{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/#primaryimage","url":"https:\/\/vr-room.ch\/wp-content\/uploads\/2022\/11\/NeRFPlayer_-A-Streamable-Dynamic-Scene-Representation-with-Decomposed-Neural-Radiance-Fields-3-52-screenshot-e1667664753387-1200x754-1.png","contentUrl":"https:\/\/vr-room.ch\/wp-content\/uploads\/2022\/11\/NeRFPlayer_-A-Streamable-Dynamic-Scene-Representation-with-Decomposed-Neural-Radiance-Fields-3-52-screenshot-e1667664753387-1200x754-1.png","width":1200,"height":754,"caption":"Nerf"},{"@type":"BreadcrumbList","@id":"https:\/\/vr-room.ch\/en\/2022\/11\/12\/nerf-player-streamt-dynamische-volumetrische-szenen-auch-fuer-vr\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Startseite","item":"http:\/\/vr-room.ch\/en\/"},{"@type":"ListItem","position":2,"name":"NeRF-Player streamt dynamische volumetrische Szenen \u2013 auch f\u00fcr VR"}]},{"@type":"WebSite","@id":"http:\/\/vr-room.ch\/en\/#website","url":"http:\/\/vr-room.ch\/en\/","name":"Schweizer Virtual Reality News","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"http:\/\/vr-room.ch\/en\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Person","@id":"http:\/\/vr-room.ch\/en\/#\/schema\/person\/1a03b3a6ced429fb4831f60fa2b37599","name":"Nathalie","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"http:\/\/vr-room.ch\/en\/#\/schema\/person\/image\/","url":"https:\/\/vr-room.ch\/wp-content\/uploads\/2017\/01\/avatar_user_12_1484159890-96x96.jpg","contentUrl":"https:\/\/vr-room.ch\/wp-content\/uploads\/2017\/01\/avatar_user_12_1484159890-96x96.jpg","caption":"Nathalie"},"url":"https:\/\/vr-room.ch\/en\/author\/nathalie\/"}]}},"_links":{"self":[{"href":"https:\/\/vr-room.ch\/en\/wp-json\/wp\/v2\/posts\/20945"}],"collection":[{"href":"https:\/\/vr-room.ch\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/vr-room.ch\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/vr-room.ch\/en\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/vr-room.ch\/en\/wp-json\/wp\/v2\/comments?post=20945"}],"version-history":[{"count":2,"href":"https:\/\/vr-room.ch\/en\/wp-json\/wp\/v2\/posts\/20945\/revisions"}],"predecessor-version":[{"id":20948,"href":"https:\/\/vr-room.ch\/en\/wp-json\/wp\/v2\/posts\/20945\/revisions\/20948"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/vr-room.ch\/en\/wp-json\/wp\/v2\/media\/20946"}],"wp:attachment":[{"href":"https:\/\/vr-room.ch\/en\/wp-json\/wp\/v2\/media?parent=20945"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/vr-room.ch\/en\/wp-json\/wp\/v2\/categories?post=20945"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/vr-room.ch\/en\/wp-json\/wp\/v2\/tags?post=20945"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}