And Then There Were Plots!

 

These plots are reproduced from Boerner et al. (2012), which utilized initial calibrations of AIA.  I have now developed enough of the response module to analyse updated instrumental properties to get my own calibrations using AIA instrument outfiles from SSW.

Filter Calibration

The transmittance is a ratio of the signal received through the instrument filters.  The values match the lowest lines modelled in Figure 2 of the paper.

 Mirror Calibration

 

This shows the reflectance of the mirrors per wavelength, and it is exactly like Figure 5 in the paper. Huzzah!

CCD Calibration  

Updated: Here the quantum efficiency of the CCD is shown per wavelength, and now it looks like Figure 6 after loading in the UV channels.

Effective Area Functions

Each effective area function displays the efficiency of the optics around the center wavelength of each instrument channel. It is calculated using the instrument calibration properties shown in the other plots.
Updated: Since re-factoring my code, I now get the shapes expected from the paper. These show the calculated effective area in blue and the effective area loaded in from the .genx file dashed in red.


***

GSoC AIA Response Functions

 

Project Details:

 

First:  A bit of a delay...

The past few weeks have been quite busy, yet rewarding. The culmination of finals week, graduation, family visiting the West coast for the first time, and finishing my senior research by doing some final remote observations have been taking up more time than anticipated. I'm sad to say that all of this stunted my progress in the GSoC project.

Then:  Back to Work!

So, in hopes to catch up, this last week I've worked with a technician on UW campus to get SSW working. Several days later, we finally just root installed ftp to get it to download. Now, I have access to an idl version of what I'm trying to redo! This has helped already to see what each piece of the code outputs.

Code:

First, I read in the idl .genx files from ssw with python.  My mentor Will sent me documentation on how to do this with scipy.io. I need some properties in these files to make the instrument response function. I was warned that this would make numpy.recarray, and have found this to be a challenging object that works with fields...? It took me a few days to figure out just how nested these objects can be!

My first function I completed was utilizing the information from the .genx files and putting them into pandas dataframes. I used each channel as a key in my next function, which is to load each channel individually.

The past few days I have been working on implementing the response function with these properties. So far, I'm having issues loading in the parameters to calculate an effective area. Mostly, it's been getting past multiplying floats and arrays. My first attempt as this has been to use numpy.ones to make everything the same size, but then the division throws errors. I am still working on it!

Goals: 

1. I made a pull request in Sunpy to make it easier for the developing team to see my work. My goal for this next week especially is to make the habit of committing code to github as I go, rather than in a big bundle like I did for this past commit. It will help to show my gradual progress, and it will help my mentors to guide the work as well.  
2. I plan to do another post this week to make up for missing one last week. It will include the pull requests that I have been working through for the past week. 
3.  Get some plots of the instrument response functions asap for midterm evaluation!!
4. Be ready and on time for the developers meeting on Wednesday, 9 am my time!!!

***

Guidance provided by Sunpy developers guide

• Found while reading documentation

• Starting to be utilized while organizing the response modules

I'm really starting to know and love the SunPy developers guide. One guideline, also recommended by Astropy:docs, is setting up a python virtual environment. This makes things easier by allowing for specific versions of packages to be used per environment.

Here's how one would start an environment using conda, which is included in the installation of Anaconda:

>>> conda create name_env python=2.7

Then, any imports for the different requirements match that python, so switching between using python 2.7 (and all of the dependencies that depend on it) to python 3.4 (with new dependencies) is as easy as pie. These commands below are how you quickly go between one environment and another.

>>> source activate name_env

>>> source deactivate

>>> source activate name2_env

Lastly, this first command makes it easy to see what's being loaded in each environment, and the second one is how to install more packages:

>>> conda list

>>> conda install name_of_package

Pretty sweet. This has already helped me in testing ChiantiPy and Sunpy, and by helping me solve a problem I ran into in the first weeks with their respective python versions. I looked into popular/new pythons and found 2.7 and 3.4 are widely used. When I researched more, I found:

- The sunpy documentation states it works best with python 2.7.
- The ChiantiPy documentation has been tested with python 3.3 specifically.

So, I chose to set up an environment with python 2.7 because I've found this loads both modules. Then, I like to run sunpy.self_test() afterwards. 

---

       I also found similar git guidelines while reading the documentation pages provided by Astropy. I've utilized the section on 'How to make code contribution' quite a bit. Last post I mentioned setting up a branch on github, and the guides I've mentioned helped with that process. Mostly, I've used git to save my own repositories, but working with the development version of a code is much more extensive.

       Sunpy has asked that each week the participants of GSoC look into one pull request and check for validity. This is a new process for me, but I welcome the practice of working in the community-developing environment. While scanning the pull requests on Sunpy, I found one that reminded me of the astropy documentation, and sparked this blog post.

Sunpy Pull Request testing: Updates to CONTRIBUTING.md #1656

• First, I recognized .md from my work in preparation for this project where I resolved an issue that led to the development of CITATION.md in Sunpy. 
• For this pull request, I read the full changes to the CONTRIBUTING.md file with and without the notes provided.

• Few things I liked:
• The instant connection to people via recommending the IRC channel. That's awesome. 
• That an example of an issue is provided. Great recommendation.
• Inputs:
• Wafels comments covered quite well any thoughts that I had.

Overall, I feel this is a well developed file that is ready to be implemented. I have not tested this yet, as the build process is unfamiliar.  Looking at the details in other developers builds has allowed me to follow their process. It seems to mirror what I've described in this blog of setting up an environment to test the file, and making sure no errors come out. I'm working through this part, and feel this is something to ask about in the SunPy IRC.

***

 

 

 

Community Bonding Period Post

Understanding what goes into AIA Response Functions

 

 

Reading:

Many of the references sent by my advisor’s for this project show interesting applications of temperature response functions that I've been scanning for details, and the science is great. One reference is the Initial Calibrations of AIA on SDO. This paper explains that the instruments on AIA do not give spectroscopic information, so there is no way to calibrate the data directly. They give the instrument-response function below, that I know I'll be utilizing:
        

"instrument-response function [R(λ,t)=Aeff(λ, t)G(λ)]"

Learning the Coding Environments:

During the community bonding period, I have taken advantage of a few resources to help me get used to the SunPy and ChiantiPy coding environments. One is the ChiantiPy tutorial. I was advised that the contribution functions are vital to this project, so I'll offer a summary of that here. This is just a small piece of what will go into the modules.

The goal is to analyse emission spectra. Chianti.core has the ability to analyse the emissivities of the three strongest lines around 200 Angstrom using a contribution function. After importing this module, I defined the temperature and density of iron 14 to use for testing and applied the contribution function.

>fe14 = ch.ion('fe_14', temperature=t, eDensity=1.e+9, em=1.e+27)

>fe14.gofnt(wvlRange=[210., 220.],top=3)

I really like how this is automatically stored in at dictionary, with keys = [‘gofnt’, ‘temperature’, ‘density’]. This will be quite beneficial later when I am running this on a number of lines

I also tested this for a different line of Iron to show the peaks in temperature (I chose 177.240) at a specific density :

> fe10 = ch.ion('fe_10', temperature=t, eDensity=1.e+9, em=1.e+27)
> fe10.gofnt(wvlRange=[170., 180.], top=3)

On a side note, I successfully made a new branch in github!

REVISION:  Thanks to the recommendations of my advisor Will Barnes, I double checked the versions of Chianti that were being loaded. As it turns out, the version I had downloaded from github was not the one being loaded. This was causing discontinuous spikes in the contribution functions that were not suppose to be there. The plots have been updated to have the right continuous shape after loading the correct ChiantiPy, and I switched to a temp of 174 to 177 for Iron 10.

 

 Goals:

For the first weeks of coding, here are my goals: 

• My first task is to implement a new response function module into sunpy/sunpy/instr. This will be on my version of sunpy in github.
• In this module, I am filling in the different classes and definitions that will need to be developed, and am implementing some documentation for each one. 
• As part of GSoC, I found out recently that I need to review a pull request in Sunpy each week as well. This will be good practice for me, and I'll try to incorporate this into the blog. 

***

GSoC Proposal Accepted!

Approved Proposal Shown here!

I'm so very excited that my GSoC project working with Sunpy has been accepted! 

In the next month, I'll be becoming more acquainted with both Sunpy and CHIANTI. Then, coding will ensue on this project. I'm also excited to be working with mentors Drew Leonard and Will Barnes.

Thanks for this opportunity Google!

***

Getting into SunPy

Part 2: Pull Request: Spectrogram log y axis #291  

 

I have found data that can be used with Sunpy's spectrogram class! A solar radio spectrometer called CALLISTO is used internationally for networking solar radio data (with direct access!). Technically, I am using the CallistoSpectrogram class in sunpy.spectra to open data that I'll use to test the Spectrogram class. Using this data, I am confident that the format loaded into the class I'd like to test is correct.

>>> import spectrogram
>>> import matplotlib.pyplot as plt
>>> from sunpy.spectra.sources.callisto import CallistoSpectrogram
>>> spectra = CallistoSpectrogram.from_file('ALASKA_20160404_000000_59.fit')
>>> gram = spectrogram.Spectrogram(spectra.data, spectra.time_axis, spectra.freq_axis, spectra.start, spectra.end)
>>> gram.plot()
>>> plt.show()

Woot, data loaded successfully!

Next, the pull request asks if: instead of the frequency being on a linear scale, is it possible to make it a log scale? Looking at the spectra.spectrogram class in sunpy, I see that the linear frequency scale took some work to implement. Simply implementing a logy keyword would be confusing with the linear keyword already in place. So, I am trying to implement the axes.set_yscale('log') after the linear scale is established. Perhaps this could be changed just before plotting for an easy addition to the class. Otherwise, I plan to implement a log frequency axis in a similar way to the linear frequency axis.

To be continued.

***

reference to helpful sunpy help page

Getting into SunPy 

Attempting Pull Request: Spectrogram log y axis #291 

 

I'm love spectra. So, when the GSoC process required that I dive into issues in the open source Python library SunPy, I looked for something regarding spectra. I'm still getting used to GitHub, though, so attempting to change the code of such a large and utilized library is kind of intimidating. Then, I found the pull request linked above using a spectrogram... Great! I have low resolution data that I can use to test this!

As it turns out, this class is still being developed. I also realized that a spectrogram is slightly different than the spectra I've been working with so far.

A spectrogram show a spectrum of frequencies as they vary with time. It is used to visualize sounds plotted with time on the x axis and frequency on the y axis. This is different, but closely related to a spectra or spectrum of light that I am accustomed to, where frequencies (or wavelengths) are plotted on the x axis and the amount of of light (or flux) is plotted on the y axis. This means that I probably shouldn't try to test it with  my visual range spectra. Instead, this is most likely working with radio frequencies.

This makes sense for solar data. Radio bursts from the corona are observed, and need to be analysed.

Getting Started:

I found the code needing to be tested. On first glance, I noticed that the spectrogram class has a plot definition, and thought that a simple 'logy' keyword and an if statement added in the plot definition would suffice, like this:
  if logy:
      ax.yscale('log')

I need to test this though, and started with loading in the module and inserting a basic 2D array of random data:

>>> import spectrogram
>>> import numpy as np
>>> data = np.ndarray((10,10), buffer = np.array(np.random.random(100)))
>>> freq_axis = np.linespace(30, 300, 10)
>>> time_axis = np.array(range(0, 10, 1))





Then, I set up some arbitrary start and end times (10 seconds) to suffice the keywords:
>>> import datetime
>>> start = datetime.datetime(2016, 2, 2, 12, 30, 30)
>>> end = datetime.datetime(2016, 2, 2, 12, 30, 40)

Put it all together:
>>> gram = spectrogram.Spectrogram(data, time_axis, freq_axis, start, end)

And, just like that I have a spectrogram object. Next, to test the plot function:
>>> gram.plot()

Colorful! Now, let's try this with my addition if statement:
>>> gram.plot(logy=True)

This flopped with many, many errors. and it is not showing a a frequency range. So, I think my data may not be in the right format.

I tried it again, this time with making a sine wave with 10 being the sampling rate.
>>> sine = np.sin(2 * np.pi * (3*10**4) * (np.arange(100)) / 10)
>>> data2 = np.ndarray((10,10), buffer=sine)
>>> gram2 = spectrogram.Spectrogram(data2, time_axis, freq_axis, start, end)

It produces:

Looking less like noise. But, still not making a frequency axis. I'm going to look for data to use so that I can reproduce the use of this module, and then add my changes to test the log y axis change.


***