In honor of my trip to Chile, I'm bringing back the blog posting, hopefully on a regular basis. More to come on the observing experience later. At the moment I have several tasks at hand in my own research that I'm behind on doing:
1. Finishing my talk and paper for the KNAC thing next weekend.
So far I have a draft of the talk power point, though it could use some work. I'm essentially using my Ann Arbor talk as a template, but I took out Alex's contributions since he'll have his own talk at KNAC and I added some background info. I'm (passively) looking for more figures to make the ppt more interesting.
2. Testing out the new supermongo install.
At this very moment I'm running Allframe on the g exposures of chip3. I'm still having some trouble with formatting issues of the output of DAOmatch that I feel like others don't have--it's necessary for me to delete columns before continuing on to DAOmaster. But so far after deleting some columns I'm up and running and Allframe has yet to crash AND it's not making crazy filenames! Having learned better, I'll wait to send the official report until I'm sure that it will finish without breaking.
If it does work, I'm going to go back and see if I can figure out the discrepancy in outputs. Even if I can't, I already have code to reformat the files to what they should be so I'm going to continue with all of the chips to create a new master list to compare with the other. I'm going to use a different exposure to reference all of the other frames to because I was having problems on some of the chips with the exposure (126) that I chose last time. I think I might go with an r exposure this time, since I don't remember having any difficulties with those.
3. Tracing back the RA and Dec of my star list to see if the astrometry could be off somewhere which now may be causing problems with calibration.
I've barely started on this but I'm going to focus more attention on it and hopefully have something figured out by the end of tonight.
4. Plot the magnitude offsets as a function of distance from the center of Willman 1 to check for a radial dependence which also could indicate problems with calibration.
I attempted this one but ran into some snags. I think there's definitely a lot more I can do after putting some more thought into it. Again, now that I'm settled in here I'll take some time during some of the longer exposures to do this--right after I've figured out the RA, Dec situation, which I think is more important at the moment.
All of this will definitely get checked off the list this week because it's already the end of September and I need to move on already.
Monday, September 21, 2009
Thursday, July 16, 2009
Positive Loglike Values!
Hallelujah.
Today:
I got the max likelihood code working with a fix to the KPNO input file. It's running at a snail's pace, though. The amoeba results look legit at first glance so I hope I'm not wasting time in letting it run.
Allframe finished last night, but I'm just noticing that all of my sharp values are 0.0. This is going to be a big problem when I go to make chi and sharp cuts to the CMD to pick out the stars. I'm looking into this and trying to run Allframe again to see if there's not an easy fix. I wasn't entirely sure what to change to fix this problem since it's so random, but I increased the Allframe iterations and I'll just see if my results are similar to what I got before.
In the meantime my post-Allframe analysis is so close and yet so far...I'm currently at a standstill with running DAOmaster on my Allframe output. For some reason it just won't read the input file. I tried the files that Ricardo sent from his run of Wil1 data and that won't run either. So I sent some of my files back to him to see if they'll run on his machine. I'm not sure if I'll be happy or not if they run there, but it'll be better than nothing. I compared my files to his and other than the sharp thing I don't see a difference. And I don't think the bad value would contribute to a read error. I'm going to take a fresh look at it a little later when I haven't been staring at the same files all day.
Tomorrow:
With any luck my max likelihood run will be done so I'll have some good Wil1 params by the end of the day.
With a huge amount of luck it won't take me another 2 weeks to figure out what's wrong with the Allframe analysis this time. Ricardo got good sharp values on the same data so I'm not sure I want to know what the problem is...but one way or another it would be awesome to have a CMD leading into the weekend. If only the allframe results would cooperate. The rest is really pretty straightforward.
In the meantime of other things running I'm going to get started on the absolute magnitude calculation. With everything else happening I doubt this will get done by the end of tomorrow. But I'll settle for by the end of Monday.
Results are finally close to coming together. I just hope it happens sooner rather than later so I can put this talk together.
Today:
I got the max likelihood code working with a fix to the KPNO input file. It's running at a snail's pace, though. The amoeba results look legit at first glance so I hope I'm not wasting time in letting it run.
Allframe finished last night, but I'm just noticing that all of my sharp values are 0.0. This is going to be a big problem when I go to make chi and sharp cuts to the CMD to pick out the stars. I'm looking into this and trying to run Allframe again to see if there's not an easy fix. I wasn't entirely sure what to change to fix this problem since it's so random, but I increased the Allframe iterations and I'll just see if my results are similar to what I got before.
In the meantime my post-Allframe analysis is so close and yet so far...I'm currently at a standstill with running DAOmaster on my Allframe output. For some reason it just won't read the input file. I tried the files that Ricardo sent from his run of Wil1 data and that won't run either. So I sent some of my files back to him to see if they'll run on his machine. I'm not sure if I'll be happy or not if they run there, but it'll be better than nothing. I compared my files to his and other than the sharp thing I don't see a difference. And I don't think the bad value would contribute to a read error. I'm going to take a fresh look at it a little later when I haven't been staring at the same files all day.
Tomorrow:
With any luck my max likelihood run will be done so I'll have some good Wil1 params by the end of the day.
With a huge amount of luck it won't take me another 2 weeks to figure out what's wrong with the Allframe analysis this time. Ricardo got good sharp values on the same data so I'm not sure I want to know what the problem is...but one way or another it would be awesome to have a CMD leading into the weekend. If only the allframe results would cooperate. The rest is really pretty straightforward.
In the meantime of other things running I'm going to get started on the absolute magnitude calculation. With everything else happening I doubt this will get done by the end of tomorrow. But I'll settle for by the end of Monday.
Results are finally close to coming together. I just hope it happens sooner rather than later so I can put this talk together.
Wednesday, July 15, 2009
Allframe works!
Today:
Got allframe working! Finally.
Compared a density plot I made of my star list with one made from the original catalog. I'm inclined to say that the one I made with the original catalog matches figure Beth sent that she made back using that catalog. But there are still significant morphological differences between my distribution from the new catalog and these other figures. This concerns me.
I also have an idea about what could be causing ridiculousness in my input file...I think it's most likely something wrong with the input file. I don't have enough time to implement some changes at the moment but I'm going to come back to this tonight and try a revised input file.
Tomorrow:
1. I want to close the loop on this stellar density plot. Hopefully I'll have Allframe results to work with.
2. Allframe will be done running, so I'll need to figure out how to put together my master list and then create a CMD.
3. Go back to working on the max likelihood stuff. I got caught up with Allframe so I'm still having many of the same issues as before. I'm wondering if I didn't quite implement Dave's corrections correctly. Something is wrong and it's probably going to take a careful eye to catch it. It would be great if I could have it running by Friday so I can be sure to have some solid results to put into my talk.
Got allframe working! Finally.
Compared a density plot I made of my star list with one made from the original catalog. I'm inclined to say that the one I made with the original catalog matches figure Beth sent that she made back using that catalog. But there are still significant morphological differences between my distribution from the new catalog and these other figures. This concerns me.
I also have an idea about what could be causing ridiculousness in my input file...I think it's most likely something wrong with the input file. I don't have enough time to implement some changes at the moment but I'm going to come back to this tonight and try a revised input file.
Tomorrow:
1. I want to close the loop on this stellar density plot. Hopefully I'll have Allframe results to work with.
2. Allframe will be done running, so I'll need to figure out how to put together my master list and then create a CMD.
3. Go back to working on the max likelihood stuff. I got caught up with Allframe so I'm still having many of the same issues as before. I'm wondering if I didn't quite implement Dave's corrections correctly. Something is wrong and it's probably going to take a careful eye to catch it. It would be great if I could have it running by Friday so I can be sure to have some solid results to put into my talk.
Monday, July 13, 2009
Today:
I Double checked the chi and sharp cuts I made:
I ended up using:
--sharp: -0.5 to 0.5
--chi: 0.8 to 1.5
This is more or less a generous cut to clean things up a bit. More could be done to get a more stringent cut. I also remade the CMD with these cuts and without getting close to Wil1. It looks pretty good. I noticed that the isochrone wasn't doing a good job of describing the data and realized that I must have used the wrong distance to Wil1 when I calculated the distance modulus. But that's fixed and all is well. I used the same code Anna and I wrote for Segue to match stars to within 1 sigma of the M92 isochrone. I also made a Hess Diagram which I've been playing with, but also looks really good.
I also set out to tweak the contour plot of stellar density:
Like I just mentioned, I did a better job of selecting the stars for the CMD and used only the stars that matched the chi/sharp and isochrone cuts but didn't do any ra/dec cuts so that I would have the whole field to work with. I messed with Beth's annulus idea only to realize that I was already scaling the smoothed density plot. But I wasn't ignoring the central region (where Wil1 is). So I made a mask that cut out the central square before I scaled things. It was a crude way of doing it and could be changed, but I think it'll do for now. My contours definitely turned out looking a lot better so I think I'm going to tweak the smoothing and binning tonight to see if I can't get it looking really good by tomorrow.
So, in all, I now have newly updated figures from KPNO data including: hess diagram, CMD with isochrone fit, chi and sharp cuts, stellar density plot.
Tomorrow:
Max likelihood for KPNO
-still dealign with negative loglikes?
-got an email from Dave--he thinks I might have to get fancy like he did with LBT. He also pointed out some other stuff in his code that might be useful to change. I'm gonna look into this more tomorrow and probably talk to him about it.
Absolute Magnitude
Beth's been thinking a lot about how to calculate this and gave me a pretty good way of going about it. I wanted to finish what I was already working on, but this is on the plate for tomorrow.
-LF
-plots and normalization
-final calculations
-mag limit from 2006--90% completion
I Double checked the chi and sharp cuts I made:
I ended up using:
--sharp: -0.5 to 0.5
--chi: 0.8 to 1.5
This is more or less a generous cut to clean things up a bit. More could be done to get a more stringent cut. I also remade the CMD with these cuts and without getting close to Wil1. It looks pretty good. I noticed that the isochrone wasn't doing a good job of describing the data and realized that I must have used the wrong distance to Wil1 when I calculated the distance modulus. But that's fixed and all is well. I used the same code Anna and I wrote for Segue to match stars to within 1 sigma of the M92 isochrone. I also made a Hess Diagram which I've been playing with, but also looks really good.
I also set out to tweak the contour plot of stellar density:
Like I just mentioned, I did a better job of selecting the stars for the CMD and used only the stars that matched the chi/sharp and isochrone cuts but didn't do any ra/dec cuts so that I would have the whole field to work with. I messed with Beth's annulus idea only to realize that I was already scaling the smoothed density plot. But I wasn't ignoring the central region (where Wil1 is). So I made a mask that cut out the central square before I scaled things. It was a crude way of doing it and could be changed, but I think it'll do for now. My contours definitely turned out looking a lot better so I think I'm going to tweak the smoothing and binning tonight to see if I can't get it looking really good by tomorrow.
So, in all, I now have newly updated figures from KPNO data including: hess diagram, CMD with isochrone fit, chi and sharp cuts, stellar density plot.
Tomorrow:
Max likelihood for KPNO
-still dealign with negative loglikes?
-got an email from Dave--he thinks I might have to get fancy like he did with LBT. He also pointed out some other stuff in his code that might be useful to change. I'm gonna look into this more tomorrow and probably talk to him about it.
Absolute Magnitude
Beth's been thinking a lot about how to calculate this and gave me a pretty good way of going about it. I wanted to finish what I was already working on, but this is on the plate for tomorrow.
-LF
-plots and normalization
-final calculations
-mag limit from 2006--90% completion
Friday, July 10, 2009
Tuesday, July 7, 2009
Super fast update because I have to go pick up the chilluns:
Allframe still doesn't work.
Got the max likelihood code working (thanks to Dave) on the SDSS data.
Almost ready to run the max likelihood code on the KPNO data--want to double check my input first.
In lieu of Allframe results, the next step with my stack is artificial star testing, right Beth? Will look at addstar tomorrow.
Also tomorrow: look more closely at max likelihood results, run the code on KPNO stars, commence the testing of artificial stars.
Allframe still doesn't work.
Got the max likelihood code working (thanks to Dave) on the SDSS data.
Almost ready to run the max likelihood code on the KPNO data--want to double check my input first.
In lieu of Allframe results, the next step with my stack is artificial star testing, right Beth? Will look at addstar tomorrow.
Also tomorrow: look more closely at max likelihood results, run the code on KPNO stars, commence the testing of artificial stars.
Wednesday, July 1, 2009
Allframe
I've been learning how to use Allframe through a lot of reading and a lot of doing.
Allframe:
Allframe needs 4 types of input files:
1) .als files for each individual exposure output by Allstar
2) .psf files for each individual exposure output by DAOphot
3) a single .mag file that DAOMaster outputs
4) a single .mch that DAOMaster outputs
Generating these:
1) .als
I used the batch_als.py code to iterate DAOphot and Allstar twice for each individual exposure. I'll use the second (better) .als files to input to Allframe. (These are suffixed with ".als" by batch_als, not to be confused with the first pass allstar outputs which are suffixed with ".als1". )
2) .psf
Again, I got this as an output from the batch_als.py run. Again, I'll use the second-pass files, which are denoted by ".psf2".
Interim Step: DAOMatch (preliminary .mch file)
These Stetson procedures are finding/matching up stars for me. I used the Allframe "cookbook" for this part, but got a lot of parameter advice from Ricardo and also referred to the "guinea pig" publication that Stetson sends out as well as the DAOphot manual.
Right now I'm testing with only the 20 exposures of Chip7. Ultimately, I'll have a .mch file for each chip which contains transformations for all 20 exposures on this chip. This .mch is NOT the file to input to Allframe, but rather a preliminary guess to be input to DAOMaster.
I'm using the following input parameters, which are subject to change but I either got from the cookbook or from Ricardo, where Ricardo's advice obviously superseded the cookbook.
DAOmatch inputs:
--Each exposure on a single chip, both bands
--output: chipN.mch
3/4) DAOMaster
The input for DAOMaster is the .mch file I generate for each chip using DAOMatch. Unfortunately, that output contains two extraneous columns (the last two) that DAOMaster doesn't like. So they have to be erased before running DAOMaster. After they're gone, the single chipN.mch file gets input into DAOMaster.
--Min, Min fraction, Enough = 1, 0.05, 1
--Maximum Sigma = 10
--Degrees of Freedom = 20 (R: 6)
--Critical Match-up Radius = 10 (decreasing by integers to 1, re-running 1 a few times til # of stars is constant)
--Assign new star IDs: y
--mean mags and scatter: y (chipN.mag)
--new transformation file: y (chipN.mch)
--all other files: n
The file that contains the mean stellar magnitudes and scatters is suffixed by .mag. That's the star list input to Allframe. The file of updated transformations between chips is suffixed by .mch and that's the final input for Allframe.
Current Status:
Beth and I took a look at Ricardo's allframe.opt file and decided to make some changes to our allframe.opt and allstar.opt files. Because I this, I have to re-run batch_als.py on all files, which is a bit time consuming.
I'm changing the options so that inner sky=2 and outer sky=20. I changed my allstar.opt to agree with Ricardo on the profile error (=0.5 as opposed to my 0.0), since I have to re-run Allstar anyway. I also used a geometric coefficient of 20 (different from Ricardo's 6) in allframe.opt because our data was taken over the course of a few nights. Maximum iterations was lowered from the default 200 to Ricardo and the cookbook's suggestion of 50 to save time. All other parameters agree with Ricardo's allframe.opt file and nothing else was changed in the allstar.opt file from before.
I'm running the batch_als.py for the 20 exposures on Chip 7 first so that it's sure to be ready for Allframing overnight. Once batch_als.py finish, I have to run DAOmatch and DAOMaster to get all 4 files for input into Allframe and then let her rip. As soon as I start everything running for Allframe, I'm going to run batch_als.py on all exposures of all chips. Once all of that is running, I'm going to return to my code that should automate DAOmatch since I never got that working properly.
Once all of Allframe input files I'll start them all Allframing, which should be ready by the time I get home tonight.
Tomorrow:
Check out my awesome Allframe results.
Figure out how to throw out all the junk Allframe gave me.
Allframe:
Allframe needs 4 types of input files:
1) .als files for each individual exposure output by Allstar
2) .psf files for each individual exposure output by DAOphot
3) a single .mag file that DAOMaster outputs
4) a single .mch that DAOMaster outputs
Generating these:
1) .als
I used the batch_als.py code to iterate DAOphot and Allstar twice for each individual exposure. I'll use the second (better) .als files to input to Allframe. (These are suffixed with ".als" by batch_als, not to be confused with the first pass allstar outputs which are suffixed with ".als1". )
2) .psf
Again, I got this as an output from the batch_als.py run. Again, I'll use the second-pass files, which are denoted by ".psf2".
Interim Step: DAOMatch (preliminary .mch file)
These Stetson procedures are finding/matching up stars for me. I used the Allframe "cookbook" for this part, but got a lot of parameter advice from Ricardo and also referred to the "guinea pig" publication that Stetson sends out as well as the DAOphot manual.
Right now I'm testing with only the 20 exposures of Chip7. Ultimately, I'll have a .mch file for each chip which contains transformations for all 20 exposures on this chip. This .mch is NOT the file to input to Allframe, but rather a preliminary guess to be input to DAOMaster.
I'm using the following input parameters, which are subject to change but I either got from the cookbook or from Ricardo, where Ricardo's advice obviously superseded the cookbook.
DAOmatch inputs:
--Each exposure on a single chip, both bands
--output: chipN.mch
3/4) DAOMaster
The input for DAOMaster is the .mch file I generate for each chip using DAOMatch. Unfortunately, that output contains two extraneous columns (the last two) that DAOMaster doesn't like. So they have to be erased before running DAOMaster. After they're gone, the single chipN.mch file gets input into DAOMaster.
--Min, Min fraction, Enough = 1, 0.05, 1
--Maximum Sigma = 10
--Degrees of Freedom = 20 (R: 6)
--Critical Match-up Radius = 10 (decreasing by integers to 1, re-running 1 a few times til # of stars is constant)
--Assign new star IDs: y
--mean mags and scatter: y (chipN.mag)
--new transformation file: y (chipN.mch)
--all other files: n
The file that contains the mean stellar magnitudes and scatters is suffixed by .mag. That's the star list input to Allframe. The file of updated transformations between chips is suffixed by .mch and that's the final input for Allframe.
Current Status:
Beth and I took a look at Ricardo's allframe.opt file and decided to make some changes to our allframe.opt and allstar.opt files. Because I this, I have to re-run batch_als.py on all files, which is a bit time consuming.
I'm changing the options so that inner sky=2 and outer sky=20. I changed my allstar.opt to agree with Ricardo on the profile error (=0.5 as opposed to my 0.0), since I have to re-run Allstar anyway. I also used a geometric coefficient of 20 (different from Ricardo's 6) in allframe.opt because our data was taken over the course of a few nights. Maximum iterations was lowered from the default 200 to Ricardo and the cookbook's suggestion of 50 to save time. All other parameters agree with Ricardo's allframe.opt file and nothing else was changed in the allstar.opt file from before.
I'm running the batch_als.py for the 20 exposures on Chip 7 first so that it's sure to be ready for Allframing overnight. Once batch_als.py finish, I have to run DAOmatch and DAOMaster to get all 4 files for input into Allframe and then let her rip. As soon as I start everything running for Allframe, I'm going to run batch_als.py on all exposures of all chips. Once all of that is running, I'm going to return to my code that should automate DAOmatch since I never got that working properly.
Once all of Allframe input files I'll start them all Allframing, which should be ready by the time I get home tonight.
Tomorrow:
Check out my awesome Allframe results.
Figure out how to throw out all the junk Allframe gave me.
Subscribe to:
Posts (Atom)