Question about your paper "A Decade's Battle on Dataset Bias: Are We There Yet?"
afpapqy opened this issue · 2 comments
I read your interesting paper "A Decade's Battle on Dataset Bias: Are We There Yet?" with great interest. You demonstrate the surprising ability of modern neural networks to classify which dataset an image comes from, even for large and diverse datasets, achieving over 80% accuracy in many cases. This is particularly impressive considering that, as a human, I find it very challenging to visually distinguish which dataset an image belongs to. The neural networks must be learning some unique characteristics to achieve such high performance.
One potential factor I'm curious about is the differences in camera intrinsic parameters (e.g., focal length, sensor size) between the devices used to capture images for different datasets. Since different datasets are often collected using various cameras/devices, this could lead to subtle but consistent differences in the images. Moreover, recent research has shown that neural networks are capable of estimating camera intrinsic parameters from images, suggesting that such information is indeed present and can be learned and utilized by the networks.
I noticed that the datasets you used come from various sources, such as Flickr, Wikipedia, web crawling, and search engines. While this diversity might introduce variations in camera parameters within each dataset, there could still be systematic differences between datasets due to the different characteristics of their sources. For example, images from Flickr and Wikipedia might have statistical differences in the types of cameras used and the photography habits of the users.
Given that humans struggle with this task while neural networks excel at it, I suspect that the networks are picking up on some unique, subtle features that are not easily discernible to the human eye. The differences in camera parameters could be one such feature.
I would be very curious to hear your thoughts on whether camera parameter differences might be a factor in dataset classification, and if you have done any analysis to investigate this possibility. If not, I believe this could be a valuable direction for further research, potentially providing new insights into the mechanisms behind the networks' ability to distinguish datasets.
Thank you again for the thought-provoking work! I look forward to your response and any additional insights you might have.
could you provide some references about neural networks capable of estimating camera intrinsic parameters from images? I am interested in it.