Setting up a lens test .... advise.
Old 01-18-2006   #1
raid
Dad Photographer
 
raid's Avatar
 
raid is online now
Join Date: Nov 2005
Location: Florida
Posts: 30,591
Setting up a lens test .... advise.

I want to do another test for the eight or nine 50mm rangefinder lenses I own. In my last (flare) test, I used Ilford XP2 Super film, and I used window light and indoor lights for illumination of the model. When setting ASA at 200, I had exposure setting f 2.0 @ 1/60 and f 4.0 @ 1/15. It was suggested that slower film would have been better for such a test. I now want to test the overall quality of each lens for portraits without shooting into the window light. Flare has now been tested and I want to move on to test something else. If I use slower film indoors, I must use a flash. Would this be OK for a lens test? Is color film also an alternative here (with flash)? The last option I see is taking the photos outdoor with slower film and natural light. This may be a little more complicated due to having 8-9 lenses and the need to keep my little model sitting without much moving around in the garden.
Your suggestion are requested.
__________________
- Raid

________________


http://raid.smugmug.com/
  Reply With Quote

Old 01-18-2006   #2
awilder
Registered User
 
awilder is offline
Join Date: May 2005
Posts: 1,351
Try using your wife as a model. She's apt to keep more still so you can be more pecise in your focus although 9 lenses would probably try her patience without the promise of jewelry. Another altenative would be a manaquin or large doll if you want to be critical in your testing.
__________________
My Gallery
  Reply With Quote

Old 01-18-2006   #3
back alley
IMAGES
 
back alley's Avatar
 
back alley is offline
Join Date: Jul 2003
Location: true north strong & free
Posts: 49,161
i think brett has a large doll you could borrow.
it wouldn't be a bother as you can blow it up upon arrival...
  Reply With Quote

Old 01-18-2006   #4
Finder
-
 
Finder is offline
Join Date: Dec 2005
Posts: 2,055
What are you testing for? I would confine the test to give results that can be compared so you can make clear judgements.
  Reply With Quote

Old 01-18-2006   #5
raid
Dad Photographer
 
raid's Avatar
 
raid is online now
Join Date: Nov 2005
Location: Florida
Posts: 30,591
Quote:
Originally Posted by Finder
What are you testing for? I would confine the test to give results that can be compared so you can make clear judgements.

I want to see which lens gives a better overall result. This could include sharpness and contrast and overall look.
__________________
- Raid

________________


http://raid.smugmug.com/
  Reply With Quote

Old 01-18-2006   #6
raid
Dad Photographer
 
raid's Avatar
 
raid is online now
Join Date: Nov 2005
Location: Florida
Posts: 30,591
Quote:
Originally Posted by awilder
Try using your wife as a model. She's apt to keep more still so you can be more pecise in your focus although 9 lenses would probably try her patience without the promise of jewelry. Another altenative would be a manaquin or large doll if you want to be critical in your testing.

Oh boy ... more jewelry is needed! I will see what I can do here. I don't like using a large doll or similar as I like to send some of the resulting photos to family and friends. I'm nor sure they would understand my use of an inflateable doll, as Joe is suggesting!
__________________
- Raid

________________


http://raid.smugmug.com/
  Reply With Quote

No flash please...
Old 01-18-2006   #7
waileong
Registered User
 
waileong is offline
Join Date: Nov 2004
Posts: 557
No flash please...

...unless you do a ceiling bounce.

Else you'll get hotspots, uneven illumination, etc.

Shoot a still subject (like a doll as suggested or a Barney doll or perhaps a flower or plant), use B mode to get the right exposure (remember reciprocity failure), bracket as necessary.

Remember that right focus and exposure are critical for evaluating resolution, contrast, look, etc. You may even want to measure the distance to the subject to confirm that your rangefinder is accurate.

Wai Leong
===
Quote:
Originally Posted by raid amin
I want to do another test for the eight or nine 50mm rangefinder lenses I own. In my last (flare) test, I used Ilford XP2 Super film, and I used window light and indoor lights for illumination of the model. When setting ASA at 200, I had exposure setting f 2.0 @ 1/60 and f 4.0 @ 1/15. It was suggested that slower film would have been better for such a test. I now want to test the overall quality of each lens for portraits without shooting into the window light. Flare has now been tested and I want to move on to test something else. If I use slower film indoors, I must use a flash. Would this be OK for a lens test? Is color film also an alternative here (with flash)? The last option I see is taking the photos outdoor with slower film and natural light. This may be a little more complicated due to having 8-9 lenses and the need to keep my little model sitting without much moving around in the garden.
Your suggestion are requested.
__________________
waileong’s Gallery
  Reply With Quote

Old 01-18-2006   #8
dgray
Registered User
 
dgray is offline
Join Date: Jan 2006
Location: Central Kansas
Posts: 94
In order for the test to actually have any significant meaning you need to eliminate as many variable from the system as possible. This would require a solid tripod and fixed subject. The slightest bit of movement from the camera or subject, especially in a portrait that is focused up close, and all of a sudden a sharp lens will be pronounced "soft." If you want some usable nice portraits, go for it, but it would be dubious to then claim the exercise a fair lens test if you are examining sharpness. With that many lenses to test you also run the risk of natural light actually changing during the process (clouds, time of day, shifting shadows...). The more controlled the environment the better. Make sure you post any results - not all of us have 9 50mm lenses to choose from and we would like to see a comparison!
  Reply With Quote

Old 01-19-2006   #9
jaapv
RFF Sponsoring Member.
 
jaapv's Avatar
 
jaapv is offline
Join Date: May 2005
Location: Hellevoetsluis, Netherlands
Posts: 8,384
I would run a test chart series beside your model shots to help you evaluate the results.
I always like the test setup DPreview uses, he only uses live models for testing flash results and inanimate subjects for the rest.
__________________
Jaap

jaapvphotography
  Reply With Quote

Old 01-19-2006   #10
Rob
rangefinder camera magnet
 
Rob is offline
Join Date: Apr 2004
Location: Atlanta, Ga. USA
Age: 61
Posts: 477
I would shoot slide film for the test. It will take away the variable of bad focus on the printers side of it and test for the best colors too.
__________________
<a href='http://www.rangefinderforum.com/photopost/showgallery.php?cat=500&ppuser=367'>My Gallery</a>
  Reply With Quote

Old 01-19-2006   #11
Nick R.
Still here
 
Nick R.'s Avatar
 
Nick R. is offline
Join Date: Dec 2004
Location: Carlstadt, NJ
Posts: 1,070
A lens' color signature is as importment as its resolution. So unless you only shoot B & W, color is a must.

One mistake I've noticed in these homegrown tests is that people tend to shoot a human face fairly centered in the field. Our minds are inately tuned to latch onto to facial images (esp. the eyes) which in turn means in such shots were apt to subconciously give less weight to details on the periphery. This leads to a bias in results toward lenses which resolve better in the center but may do poorly on the edges over ones which may do a better job over the whole field. In other words, shoot something of interest over the whole frame of your negative.
__________________
I'll come running to tie your shoe
My Gallery
  Reply With Quote

Old 01-19-2006   #12
Ben Z
Registered User
 
Ben Z is offline
Join Date: Jun 2005
Posts: 2,552
I think the method all depends on your reasons for testing. If you're looking to publish the results on the internet and that they should have relevance for others, you need to be as universal as possible since everybody has different photographic needs. Then again, from what I've observed, no matter what you do, the people who own a particular lens that scores lower on your test will belittle your test methodology and perhaps even challenge your intelligence. OTOH if the test is to have significance for your own photography then I'd say to use subject, film, lighting, aperture etc. as you anticipate shooting in real life, because the results will be more relevant to you.

As an example, I had a chance to test the new 50 Summilux-ASPH against my pre-ASPH. There's only one reason I would pay extra and carry the weight of a 50/1.4 vs a Cron, and that's to shoot in very low light, handheld, on fast b&w film. So my side-by side shots were made in a jazz bar, at about 15 ft, at f/1.4 and 1/8 sec, on ISO 400 b&w, and there was absolutely no detectable difference in sharpness or contrast anywhere in the frame. Had I shot these lenses together on a tripod using slow slide film with a planar subject with fine detail throughout, I might have seen where the improvements were made to the ASPH, but since I would never shoot such a lens of that kind of subject (I'd use a Hasselblad if I wanted maximum fine detail), such a test might have been useful to justify my having spent $2500 on that lens, but would have had no relevance to how I would be using it.
  Reply With Quote

Old 01-19-2006   #13
raid
Dad Photographer
 
raid's Avatar
 
raid is online now
Join Date: Nov 2005
Location: Florida
Posts: 30,591
It lis much easier for me to replace a live model by some object that is not moving at all, so this advice by Weileong certainly will be followed.

I also will follow Nick's advice in using both color and B&W film. This is not a problem at all. I shoot a soften color as I shoot with B&W. I will try to remember not to center the object of interest and set it slightly to one side of the frame.

As for Japp's suggestion to use a test chart, I must confess that I don't have one and never checked out before where online to get one. My own "test chart" was a peice of white paper with many letters on it, similar to an eye sight test but more home made.

I often use slide film, but scanning all the slides will be time consuming, and I rather use a method in which I can ask a commercial lab to provide me with scans.

I agree with Ben's comment above. In my recent test for eight 50mm lenses in which I tested for flare, the results of the Summicron (rigid first version) were below what is expected of a Summicron lens. My camera repair person told me yesterday that the amount of haze he found was not much, and he still has to clean it. Most people commenting on the test result were surprised by the flare and they suggested that I check the lens out for internal haze.

I rarely see a test for 8-9 similar lenses, and this is my main gaol here. I want to see if I can find real differences between some of the lenses used in the test, and I also enjoy reading your comments since different photographers may have different preferences. Sometimes certain small things are pointed out by some people that I did not even notice.
__________________
- Raid

________________


http://raid.smugmug.com/
  Reply With Quote

Old 01-21-2006   #14
Simon Larbalestier
Registered User
 
Simon Larbalestier's Avatar
 
Simon Larbalestier is offline
Join Date: Oct 2004
Location: Bangkok, Thailand
Age: 57
Posts: 1,112
Ben that one of the best written posts about practical lens testing i've read on the internet forums.
Thank you.

Simon
  Reply With Quote

Old 01-21-2006   #15
raid
Dad Photographer
 
raid's Avatar
 
raid is online now
Join Date: Nov 2005
Location: Florida
Posts: 30,591
Yes, practicality is the most important factor when it comes to lens testing. How practical is the test and how realistic the findings? If I shoot mostly handheld then the test should be also this way even though I may induce some variation due to handshake and related issues, but the test results would reflect more accurately how the lenses will be used. Replcation in such a case is very important to obtain more validity.
__________________
- Raid

________________


http://raid.smugmug.com/
  Reply With Quote

Old 01-21-2006   #16
FrankS
Registered User
 
FrankS's Avatar
 
FrankS is offline
Join Date: Aug 2004
Location: Canada, eh.
Age: 62
Posts: 19,391
Quote:
Originally Posted by Ben Z
I think the method all depends on your reasons for testing. If you're looking to publish the results on the internet and that they should have relevance for others, you need to be as universal as possible since everybody has different photographic needs. Then again, from what I've observed, no matter what you do, the people who own a particular lens that scores lower on your test will belittle your test methodology and perhaps even challenge your intelligence. OTOH if the test is to have significance for your own photography then I'd say to use subject, film, lighting, aperture etc. as you anticipate shooting in real life, because the results will be more relevant to you.

As an example, I had a chance to test the new 50 Summilux-ASPH against my pre-ASPH. There's only one reason I would pay extra and carry the weight of a 50/1.4 vs a Cron, and that's to shoot in very low light, handheld, on fast b&w film. So my side-by side shots were made in a jazz bar, at about 15 ft, at f/1.4 and 1/8 sec, on ISO 400 b&w, and there was absolutely no detectable difference in sharpness or contrast anywhere in the frame. Had I shot these lenses together on a tripod using slow slide film with a planar subject with fine detail throughout, I might have seen where the improvements were made to the ASPH, but since I would never shoot such a lens of that kind of subject (I'd use a Hasselblad if I wanted maximum fine detail), such a test might have been useful to justify my having spent $2500 on that lens, but would have had no relevance to how I would be using it.
But Ben and Raid, under these uncontrolled conditions, (especially the handheld shots at low speeds), perhaps the $40 Jupiter 8 lens will have faired even better, if at the moment of exposure, your hands happened to hold the camera slightly more still during that exposure.

Coming from a science background as I do, I have to take exception to your basic premise of lens testing. If you are going to bother testing lenses against each other, you HAVE to eliminate all variables other than the lens being tested otherwise you cannot come to a conclusion about the performance of a lens where many variables have affected the outcome.

The idea of "real world shooting conditions" just doesn't work if you are testing lenses. Either you understand that or you don't, but that is the way it is, and I stand behind that statement 100%, with all of my integrity on display for ridicule if it is incorrect.

Raid, as a statistician, I am somewhat surprised that this is not obvious to you.
__________________
my little website: http://frankfoto.jimdo.com/

photography makes me happy

Last edited by FrankS : 01-21-2006 at 20:19.
  Reply With Quote

Old 01-21-2006   #17
Simon Larbalestier
Registered User
 
Simon Larbalestier's Avatar
 
Simon Larbalestier is offline
Join Date: Oct 2004
Location: Bangkok, Thailand
Age: 57
Posts: 1,112
I think there's also a subjective more emotional issue here to do with the cost of certain lenses.
I posted a WTB ad on another site and it fueled a very interesting debate about the pros and cons of pre ASAP and ASAP 35 luxes. I was only looking for someone selling one! I like many people, i'm sure, read a lot of reviews and comments posted by users on many forums and although i agee with Frank's scientific approach and others like Erwin's lens reviews when it comes to evaluating lenses using established benchmarks to show different lenses characteristics,i'm also fascinated by Dante Stella's reviews of lenses on his very imformative site which seems to base evaluations based on the way they perform when used in the field ( he certainly illustrates it with great examples). I'm ot really sure if "emotional" is exactly the right word here but it certainly a kick in the lower regions when you've shelled out a large amount for the latest Leica ASAP (for example) and then discovered when you're printing your negs you can't tell the difference! Sure it feels good to know the lens you just invested in is worth the expense and hits the mark in latest technological lens developments but often when the negs are all printed and evaluated its the very moment captured in the photograph that makes it a winner and as is often the case with me!
What counts then is whatever's on the camera. I think i've rambled on a bit and time to get back to scanning my negs........

Simon

Last edited by Simon Larbalestier : 01-21-2006 at 20:51.
  Reply With Quote

Old 01-21-2006   #18
FrankS
Registered User
 
FrankS's Avatar
 
FrankS is offline
Join Date: Aug 2004
Location: Canada, eh.
Age: 62
Posts: 19,391
There's a world of difference between a lens's potential and the perfection of a moment captured by the photographer. You can demonstrate one under controlled conditions, the other is totally up to the photographer's skill.
__________________
my little website: http://frankfoto.jimdo.com/

photography makes me happy
  Reply With Quote

Old 01-21-2006   #19
Simon Larbalestier
Registered User
 
Simon Larbalestier's Avatar
 
Simon Larbalestier is offline
Join Date: Oct 2004
Location: Bangkok, Thailand
Age: 57
Posts: 1,112
Hi Frank
yes you're right - i think i was just trying to say that what sometimes fuels my GAS attacks are reviews which make you want to go and buy the latest lens version - then i have to control myself and think how i print my images and delete my saved searches on Ebay!
Simon
  Reply With Quote

Old 01-21-2006   #20
FrankS
Registered User
 
FrankS's Avatar
 
FrankS is offline
Join Date: Aug 2004
Location: Canada, eh.
Age: 62
Posts: 19,391
Simon, and then the issue becomes more confused if, as I do, one searches for classic, vintage results in lenses as opposed to simply sharper.
__________________
my little website: http://frankfoto.jimdo.com/

photography makes me happy
  Reply With Quote

Old 01-21-2006   #21
raid
Dad Photographer
 
raid's Avatar
 
raid is online now
Join Date: Nov 2005
Location: Florida
Posts: 30,591
Quote:
Originally Posted by FrankS
But Ben and Raid, under these uncontrolled conditions, (especially the handheld shots at low speeds), perhaps the $40 Jupiter 8 lens will have faired even better, if at the moment of exposure, your hands happened to hold the camera slightly more still during that exposure.

Coming from a science background as I do, I have to take exception to your basic premise of lens testing. If you are going to bother testing lenses against each other, you HAVE to eliminate all variables other than the lens being tested otherwise you cannot come to a conclusion about the performance of a lens where many variables have affected the outcome.

The idea of "real world shooting conditions" just doesn't work if you are testing lenses. Either you understand that or you don't, but that is the way it is, and I stand behind that statement 100%, with all of my integrity on display for ridicule if it is incorrect.

Raid, as a statistician, I am somewhat surprised that this is not obvious to you.
Frank,
If you take many photos at the same setting with one camera, then even handheld photos may give you an insight of its performance. Don't you agree with this? Being a statistician, my professional live has been all about quality control and experiments, so I would not do this with a "scientifically planned experiement". I agree with you on this issue.
__________________
- Raid

________________


http://raid.smugmug.com/
  Reply With Quote

Old 01-21-2006   #22
raid
Dad Photographer
 
raid's Avatar
 
raid is online now
Join Date: Nov 2005
Location: Florida
Posts: 30,591
I certainly will have a controlled experiment in my test, and this includes all factors I can think of that I can fix for all lenses used. I guess, the hand-held in the field testing of lenses is more a personal matter. In my last test, I had three replications, and I am quite confident that my test results overall hold up. My lenses were either recently CLA'd or looked very clean and did well when used recently. The only lens I took to a camera repair person to check out for internal haze was the Summicron (rigid). Even that one may actually be quite clean internally, but I still have the camera repairman clean it anyways.
__________________
- Raid

________________


http://raid.smugmug.com/
  Reply With Quote

Old 01-21-2006   #23
Simon Larbalestier
Registered User
 
Simon Larbalestier's Avatar
 
Simon Larbalestier is offline
Join Date: Oct 2004
Location: Bangkok, Thailand
Age: 57
Posts: 1,112
That's very true Frank, in fact that's exactly why i purchased a Nikkor 3.5/2.5 for my S3 and am looking for the early 35 1.4 lux. One day i will have to stop.
OT i find myself searching for vintage bags like old fishing bags that are well worn......

Simon
  Reply With Quote

Old 01-21-2006   #24
FrankS
Registered User
 
FrankS's Avatar
 
FrankS is offline
Join Date: Aug 2004
Location: Canada, eh.
Age: 62
Posts: 19,391
In my opinion, the greatest and limiting factor in lens performance is camera movement during exposure. To test and compare the performance of different lenses without controlling this variable, again in my opinion, is senseless. Simple as that.
__________________
my little website: http://frankfoto.jimdo.com/

photography makes me happy
  Reply With Quote

Old 01-21-2006   #25
raid
Dad Photographer
 
raid's Avatar
 
raid is online now
Join Date: Nov 2005
Location: Florida
Posts: 30,591
I used a heavy tipod and also a cable release in order to reduce vibration and camera movement. This also allowed me to focus more accurately.
__________________
- Raid

________________


http://raid.smugmug.com/
  Reply With Quote
Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Leica M lens test MP Guy Leica M Film Cameras 10 11-25-2005 18:16
Voigtlander lens test reports satbunny Voigtlander Bessa Leica Mount Cameras 14 07-31-2005 19:54
hello, and two questions about the 7s stet Canon Leica Screw Mount Film Rangefinders 46 04-29-2005 01:41
35mm lens test zuikologist Rangefinder Photography Discussion 24 12-22-2004 23:43
Mamiya announces 50mm f/4.5 L Lens for M7 MP Guy Rangefinder Photography Discussion 0 07-31-2003 08:34



All times are GMT -8. The time now is 11:39.


vBulletin skin developed by: eXtremepixels
Powered by vBulletin® Version 3.6.8
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.

All content on this site is Copyright Protected and owned by its respective owner. You may link to content on this site but you may not reproduce any of it in whole or part without written consent from its owner.