Inkblot Illusion

Inkblot Illusion
Inkblot on the blind curtain expands.
DOI: https://doi.org/10.6084/m9.figshare.6137582
Eiji Watanabe, Inkblot Illusion, 2018


Illusory Motion Reproduced by Deep Neural Networks Trained for Prediction

Deep neural networks (DNNs), which have been developed with reference to the network structures and the operational algorithms of the brain, have achieved notable success in a broad range of fields, including computer vision, in which they have produced results comparable to, and in some cases superior to, human experts. In recent years, DNNs have also been expected to be useful as a tool for studies of the brain.

Recently a research team led by associate professor Eiji Watanabe of the National Institute for Basic Biology successfully reproduced illusory motion by DNNs trained for prediction.

The DNNs are based on predictive coding theory (Figure 1), which assumes that the internal models of the brain predict the visual world at all times and that errors between the prediction and the actual sensory input further refine the internal models. If the theory substantially reproduces the visual information processing of the brain, then the DNNs can be expected to represent the human visual perception of motion.

In this research, the DNNs were trained with natural scene videos of motion from the point of view of the viewer (Figure 2), and the motion prediction ability of the obtained computer model was verified using a rotating propeller in unlearned videos and the “Rotating Snake Illusion” (Figure 3). The computer model accurately predicted the magnitude and direction of motion of the rotating propeller in the unlearned videos. Surprisingly, it also represented the rotational motion for illusion images that were not moving physically, much like human visual perception (Figure 4). While the trained network accurately reproduced the direction of illusory rotation, it did not detect motion components in negative control pictures wherein people do not perceive illusory motion.

This research supports the exciting idea that the mechanism assumed by the predictive coding theory is a basis of motion illusion generation. Using sensory illusions as indicators of human perception, deep neural networks are expected to contribute significantly to the development of brain research.

These research results were published in Frontiers in Psychology on March 15. This research was conducted as a collaborative research project by the National Institute for Basic Biology, SOKENDAI (the Graduate University for Advanced Studies), Ritsumeikan University, the National Institute for Physiological Sciences, and Sakura Research Office.

Figure 1: A schematic diagram of PredNet (a modification of Figure 1 in Lotter et al. 2016, arXiv:1605.08104). Illustration of information flow within a single layer is presented. Vertical arrows represent connections with other layers. Each layer consists of “Representation” neurons, which output a layer-specific “Prediction” at each time step, which is subtracted from “Target” to produce an error, which is then propagated laterally and vertically in the network. External data or a lower-layer error signal is input to “Target”. In each layer, the input information is not processed directly, and the prediction error signal is processed.

Figure 2: Training videos. Models were trained using videos from the First-Person Social Interactions Dataset (Fathi, et al, 2012, CVPR12@Providence) which contains day-long videos of eight subjects spending their day at Disney World Resort in Orlando, Florida. The cameras were mounted on a cap worn by the subjects.

Figure 3: Akiyoshi Kitaoka’s rotating snake illusions (the left panel). People perceive clockwise or counter-clockwise motion depending on colour alignment. Negative controls (non-illusions) for which people perceive no motion are presented in the right panel. To experience stronger illusory motion perception, please refer to “Akiyoshi’s illusion pages”, http://www.ritsumei.ac.jp/~akitaoka/index-e.html.

Figure 4: The DNNs detected optical flow vectors in the illusion. Optical flow vectors detected between a pair of consecutive predictive images of the illusion. Red bars denote the direction and magnitude of vectors, yellow dots denote the start points of the vectors. The left is a single ring of the rotating snake illusion, and the right is a negative control image. 

Frontiers in Psychology (2018) Volume 9, Article 345.
“Illusory Motion Reproduced by Deep Neural Networks Trained for Prediction” Eiji Watanabe, Akiyoshi Kitaoka, Kiwako Sakamoto, Masaki Yasugi and Kenta Tanaka
DOI: 10.3389/fpsyg.2018.00345




Discovery of dynamic seasonal changes in color perception ~The small fish "medaka" shows large differences in color perception in summer and winter~

Medaka Fish in Summer and Winter (from NIBB HP)
In many areas, the environment fluctuates greatly depending on the season, and animals living in those areas must adapt to the changing environment. A research group from the National Institute for Basic Biology and Nagoya University in Japan found that color perception of Medaka, a small fish inhabiting rice fields and streams, varies greatly according to seasonal changes. Collaborative Research Projects with Yoshimura Laboratory. For details, please refer to NIBB's website.

The article “Dynamic plasticity in phototransduction regulates seasonal changes in color perception” was published in Nature Communications.


Tsuyoshi Shimmura, Tomoya Nakayama, Ai Shinomiya, Shoji Fukamachi, Masaki Yasugi, Eiji Watanabe, Takayuki Shimo, Takumi Senga, Toshiya Nishimura, Minoru Tanaka, Yasuhiro Kamei, Kiyoshi Naruse, Takashi Yoshimura


3DCG animations for studying social approach behaviour in medaka fish

We studied social approach behaviour in medaka fish using three-dimensional computer graphic (3DCG) animations based on the morphological features and motion characteristics obtained from real fish. This is the first study which used 3DCG animations and examined the relative effects of morphological and motion cues on social approach behaviour in medaka. Various visual stimuli, e.g., lack of motion, lack of colour, alternation in shape, lack of locomotion, lack of body motion, and normal virtual fish in which all four features (colour, shape, locomotion, and body motion) were reconstructed, were created and presented to fish using a computer display. Medaka fish presented with normal virtual fish spent a long time in proximity to the display, whereas time spent near the display was decreased in other groups when compared with normal virtual medaka group. The results suggested that the naturalness of visual cues contributes to the induction of social approach behaviour. Differential effects between body motion and locomotion were also detected. 3DCG animations can be a useful tool to study the mechanisms of visual processing and social behaviour in medaka.

3DCG model of Medaka Fish
Tomohiro Nakayasu, Masaki Yasugi, Soma Shiraishi, Seiichi Uchida, Eiji Watanabe, Three-dimensional computer graphic animations for studying social approach behaviour in medaka fish: Effects of systematic manipulation of morphological and motion cues. PLOS ONEhttps://doi.org/10.1371/journal.pone.0175059 (2017)

Tracking software (Medaka Tracker), 3D polygon models, and 3D animations are freely available.

Video Demo of 3D animations:


Shelf-Shadow Illusion

Upward shadows look darker than downward shadows.
DOI: https://doi.org/10.6084/m9.figshare.6137558.v1
Eiji Watanabe, Shelf-Shadow Illusion, 2013

Four identical Shelves with shadows

Shelf-Shadow Illusion,  "Okazaemon" Version

Shelf-Shadow Illusion, Photograph Version


Biological motion stimuli are attractive to medaka fish

In many social fish species, visual cues play an important role in inducing shoaling behavior. The present study is the first to examine whether and how "biological motion" depicting a moving creature by means of only a small number of isolated points induces shoaling behaviour in fish. Medaka (Oryzias latipes) were used because they are known to have high visual acuity and exhibit a strong tendency to form shoals. In experiment 1, we found that the presentation of medaka biological motion resulted in heightened shoaling behaviour when compared with that of non-biological motion (depicted by a small number of points placed at fixed distances that moved at a constant speed in a constant direction). In experiment 2, it was indicated that medaka biological motion was more effective at inducing shoaling behaviour when compared with human biological motion. In experiment 3, it was demonstrated that shoaling behaviour was largely dependent on the smoothness of the biological motion. In experiment 4, we revealed that shoaling behaviour was maximised in normal speed group and decreased in faster- and slower-than-normal speed groups. In experiment 5, it was shown that shoaling behaviour was slightly reduced when a reversed movie was presented. These results suggest that motion information extracted from conspecifics was sufficient to induce shoaling behaviour in medaka and that deviation from normal and familiar motion impeded shoaling behaviour. The naturalness of motion may be responsible for the induction of shoaling behaviour.

Nakayasu, T. & Watanabe, E. Biological motion stimuli are attractive to medaka fish. Animal Cognition 2, 219 (2013).

URL of the paper (Open Access)


Biological Motion of Medaka Fish (Demo Video)


Visual motion with pink noise induces predation behaviour

Visual motion cues are one of the most important factors for eliciting animal behaviour, including predator-prey interactions in aquatic environments. To understand the elements of motion that cause such selective predation behaviour, we used a virtual plankton system where the predation behaviour in response to computer-generated prey was analysed. First, we performed motion analysis of zooplankton (Daphnia magna) to extract mathematical functions for biologically relevant motions of prey. Next, virtual prey models were programmed on a computer and presented to medaka (Oryzias latipes), which served as predatory fish. Medaka exhibited predation behaviour against several characteristic virtual plankton movements, particularly against a swimming pattern that could be characterised as pink noise motion. Analysing prey-predator interactions via pink noise motion will be an interesting research field in the future.

Matsunaga, W. & Watanabe, E. Visual motion with pink noise
induces predation behaviour. Scientific Reports 2, 219 (2012).

[HTML site]

[pdf download site]

[How to make a Medaka Fish Toy, YouTube Video]


Flash-drag Effect: Optical Illusion 3D

We have uploaded "Flash-drag Effect: Optical Illusion 3D".

The video is available in YouTube (click URL described in the next line).

Further, a demonstration animation of depth perception has been uploaded.

The video is also available in YouTube.

& one more optical illusion (Geared flash-lag effect, 01/06/2012).