[895] in I/T Delivery
Re: Data Warehouse Delivery monthly report
daemon@ATHENA.MIT.EDU (Mary Weisse)
Thu Oct 3 20:48:19 2002
Message-Id: <4.3.2.7.2.20021003204424.01779790@hesiod>
Date: Thu, 03 Oct 2002 20:50:31 -0400
To: Robert Ferrara <rferrara@mit.edu>, delivery@mit.edu
From: Mary Weisse <mca@MIT.EDU>
Cc: thorne@mit.edu
In-Reply-To: <5.1.0.14.2.20020930190638.01e52e80@po9.mit.edu>
Mime-Version: 1.0
Content-Type: multipart/alternative;
boundary="=====================_133805612==_.ALT"
--=====================_133805612==_.ALT
Content-Type: text/plain; charset="iso-8859-1"; format=flowed
Content-Transfer-Encoding: quoted-printable
Team Name: Data Warehouse
Team Leader: Scott Thorne
Report Date: October 3, 2002
URL: http://web.mit.edu/warehouse/
This month's accomplishments (September 2002)
=B7 AAUDE Data Warehouse -- completed development work on Phase II and=
=20
Phase III.
o Load new data for AAUP Faculty Salary
o Full re-load of all data (1979-2000 plus the new year, 2001) in=
=20
fact tables for NSF
Graduate Student Support and R&D Expenditures
o Web metadata maintenance
=B7 Prepared AAUDE training at MIT in October.
=B7 Work on migrating warehouse Web/CGI applications from SNOWDROP to=
=20
BLUE-GOOSE. Worked with Steve Turner to set up a virtual host on BLUE-GOOSE=
=20
to accomplish this. Changed configuration file. Modified programs to point=
=20
to the new server, etc. Still testing.
=B7 Reviewed all the components of Graduate Admissions data in the=
Data=20
Warehouse with Admissions.
=B7 Completed work to refine Graduate Admissions tables and stars. Set=
=20
up and scheduled weekly and monthly processes to refresh the tables.
=B7 Created sample Graduate Admissions queries for Admissions.
=B7 Continued discussions with SSIT, Registrar's Office, and Sloan=20
(Jean Dzengeleski) regarding the loading of student tuition assessment data=
=20
from MITSIS to Data Warehouse for the SSM Tuition Revenue Sharing project.
=B7 Began analysis and data modeling of MIT Medical summary data.
=B7 Continued work on Freewilly setup and work on Oracle 8.1.7=
migration.
o Compiled table size, table type, load type, load frequency,=20
objects, etc. for all
WAREUSER tables in the database.
o Wrote scripts to move files between systems during migration=20
period.
o Meet weekly with Jim Goode to discuss database issues.
o Finished development and testing of Pro*C program to generate=20
'create table' scripts.
=B7 Moved LDS personnel-to-cost collector data into the warehouse for=
=20
EH&S project.
=B7 Implemented the mechanism to update check encashment date and void=
=20
check information into the warehouse (SAP data).
=B7 Began analysis and data modeling of TLO data..
=B7 Worked with FSS and CAO to correct issues with clearing data in=
the=20
warehouse (analysis and programming work).
=B7 Started the analysis of moving Annual Forecast Budget into the=20
warehouse.
=B7 Finished all development of full and incremental load of=20
whlibrary_item_detail table on Freewilly.
o Created test data - 40000 records randomly selected=20
representing different
Open Date Periods starting from 1995 to present.
o Released the table with test data to the user for further=20
reviewing/testing on ORCA
o Developed, presented and distributed to the user accompanying=20
documentation
describing each field of the newly created "Item Detail=
Star".
o Identified various exiting data issues and addressed it to the=
=20
user.
Based on the user's feedback worked on revisiting the Item=
=20
Table
( e.x. addition of some new fields).
o Provided end user training.
=B7 Continue working with ASST and DS to get a new Production DW=20
machine (Freewilly) up and running.
=B7 Continued to work with ASST and DS to get a new make the current=
=20
Orca the new test warehouse machine.
=B7 Migrate all Warehouse Web/CGI applications from SNOWDROP and ELSA=
=20
to BLUE-GOOSE.
=B7 Continue hiring process (Replace Steve Turner who is transitioning=
=20
out of DW as of June 30).
=B7 Complete Financial Aid Federal Work Study
=B7 Continue to work on the analysis of the SAP/HR implementation in=
=20
the warehouse.
=B7 Continue design and implementation of new HR Org hierarchy.
=B7 Begin data modeling sessions with MIT Medical.
=B7 Begin data modeling sessions with TLO.
=B7 Begin data modeling sessions with the MIT Card Office
=B7 Continue data work with the Project Intersection team.
=B7 Begin design of the Library Subscription data.
=B7 Continued Query tool/web tool investigation.
=B7 Continued to work on the analysis of the SAP/HR implementation in=
=20
the warehouse.
=B7 Continued design and implementation of new HR Org hierarchy.
=B7 Continue work with the Project Intersection team.
Next Month's Plans (October, 2002)
=B7 Continued work on Freewilly setup and work on Oracle 8.1.7=
migration.
o Continue working on Pro*C program to generate index scripts=20
with new
tablespace names and correct new 'initial,'next'=20
parameters. Need
to develop part of the process that will generate these=20
scripts for ALL tables for a particular user. o o Identify all components=
=20
to be moved (all conversions involved, all jobs and all scripts).
=B7 Continue work on migrating warehouse Web/CGI applications from=20
SNOWDROP to BLUE-GOOSE.
=B7 Fully productionalize Graduate Admissions (add metadata to web=
page).
=B7 Load student tuition assessment data.
=B7 Continue working on moving complete Library piece to Freewilly.
=B7 Prepare for AAUDE board meeting in Washington DC.
=B7 Continue hiring process (Replace Steve Turner who is transitioning=
=20
out of DW as of June 30).
=B7 Continue to work on the analysis of the SAP/HR implementation in=
=20
the warehouse.
=B7 Continue design and implementation of new HR Org hierarchy.
=B7 Continue data modeling sessions with MIT Medical.
=B7 Continue data modeling sessions with TLO.
=B7 Begin data modeling sessions with the MIT Card Office
=B7 Continue data work with the Project Intersection team.
=B7 Begin design of the Library Subscription data.
=B7 Continued Query tool/web tool investigation.
Next Community Milestones:
=B7 Migration to new server and upgrade database from Orcal 7.3.4 to=
=20
8.1.7 -- November 2002
=B7 Rollout of the SAP/HR/CYBORG - Early 2003
=B7 Load summary (by service) MIT Medical data into the DW.
=B7 Load TLO case data into the DW.
=B7 Load MIT Card Office data (including pictures) into the DW.
--=====================_133805612==_.ALT
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<html>
<b>Team Name</b>: Data Warehouse<br>
<b>Team Leader:</b> Scott Thorne<br>
<b>Report Date:</b> October 3, 2002<br>
<b>URL:</b>
<a href=3D"http://web.mit.edu/warehouse/" eudora=3D"autourl"><font=
color=3D"#0000FF"><u>http://web.mit.edu/warehouse/</a><br>
<br>
<br>
</u></font><b>This month's accomplishments (September 2002)<br>
</b><font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>AAUDE Data Warehouse -- completed development work on Phase II and=
Phase III.<br>
<font face=3D"Courier New,=
Courier"><x-tab> </x-tab>o&n=
bsp; </font>Load new data for AAUP Faculty Salary<br>
<font face=3D"Courier New,=
Courier"><x-tab> </x-tab>o&n=
bsp; </font>Full re-load of all data (1979-2000 plus the new year, 2001) in=
fact tables for NSF<br>
<x-tab> </x-tab><x-tab> =
</x-tab>Graduate Student Support=
and R&D Expenditures<br>
<font face=3D"Courier New,=
Courier"><x-tab> </x-tab>o&n=
bsp; </font>Web metadata maintenance<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Prepared AAUDE training at MIT in October.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Work on migrating warehouse Web/CGI applications from SNOWDROP to=
BLUE-GOOSE. Worked with Steve Turner to set up a virtual host on BLUE-GOOSE=
to accomplish this. Changed configuration file. Modified programs to point=
to the new server, etc. Still testing.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Reviewed all the components of Graduate Admissions data in the Data=
Warehouse with Admissions. <br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Completed work to refine Graduate Admissions tables and stars. Set=
up and scheduled weekly and monthly processes to refresh the tables. <br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Created sample Graduate Admissions queries for Admissions.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continued discussions with SSIT, Registrar's Office, and Sloan (Jean=
Dzengeleski) regarding the loading of student tuition assessment data from=
MITSIS to Data Warehouse for the SSM Tuition Revenue Sharing project.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Began analysis and data modeling of MIT Medical summary data.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continued work on Freewilly setup and work on Oracle 8.1.7=
migration. <br>
<font face=3D"Courier New,=
Courier"><x-tab> </x-tab>o&n=
bsp; </font>Compiled table size, table type, load type, load frequency,=
objects, etc. for all <br>
<x-tab> </x-tab><x-tab> =
</x-tab>WAREUSER tables in the=
database. <br>
<font face=3D"Courier New,=
Courier"><x-tab> </x-tab>o&n=
bsp; </font>Wrote scripts to move files between systems during migration=
period. <br>
<font face=3D"Courier New,=
Courier"><x-tab> </x-tab>o&n=
bsp; </font>Meet weekly with Jim Goode to discuss database issues.<br>
<font face=3D"Courier New,=
Courier"><x-tab> </x-tab>o&n=
bsp; </font>Finished development and testing of Pro*C program to generate=
'create table' scripts.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Moved LDS personnel-to-cost collector data into the warehouse for=
EH&S project. <br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Implemented the mechanism to update check encashment date and void=
check information into the warehouse (SAP data). <br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Began analysis and data modeling of TLO=
data..<x-tab> </x-tab><br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Worked with FSS and CAO to correct issues with clearing data in the=
warehouse (analysis and programming work). <br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Started the analysis of moving Annual Forecast Budget into the=
warehouse. <br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Finished all development of full and incremental load of=
whlibrary_item_detail table on Freewilly.<br>
<font face=3D"Courier New,=
Courier"><x-tab> </x-tab>o&n=
bsp; </font>Created test data - 40000 records randomly selected representing=
different <br>
<x-tab> </x-tab><x-tab> =
</x-tab>Open Date Periods starting=
from 1995 to present. <br>
<font face=3D"Courier New,=
Courier"><x-tab> </x-tab>o&n=
bsp; </font>Released the table with test data to the user for further=
reviewing/testing on ORCA <br>
<font face=3D"Courier New,=
Courier"><x-tab> </x-tab>o&n=
bsp; </font>Developed, presented and distributed to the user accompanying=
documentation <br>
<x-tab> </x-tab><x-tab> =
</x-tab>describing each field of=
the newly created "Item Detail Star".<br>
<font face=3D"Courier New,=
Courier"><x-tab> </x-tab>o&n=
bsp; </font>Identified various exiting data issues and addressed it to the=
user. <br>
<x-tab> </x-tab><x-tab> =
</x-tab>Based on the user's=
feedback worked on revisiting the Item Table <br>
<x-tab> </x-tab><x-tab> =
</x-tab>( e.x. addition of some=
new fields).<br>
<font face=3D"Courier New,=
Courier"><x-tab> </x-tab>o&n=
bsp; </font>Provided end user training.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continue working with ASST and DS to get a new Production DW machine=
(Freewilly) up and running.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continued to work with ASST and DS to get a new make the current=
Orca the new test warehouse machine.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Migrate all Warehouse Web/CGI applications from SNOWDROP and ELSA to=
BLUE-GOOSE.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continue hiring process (Replace Steve Turner who is transitioning=
out of DW as of June 30).<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Complete Financial Aid Federal Work Study<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continue to work on the analysis of the SAP/HR implementation in the=
warehouse.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continue design and implementation of new HR Org hierarchy.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Begin data modeling sessions with MIT Medical.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Begin data modeling sessions with TLO.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Begin data modeling sessions with the MIT Card Office<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continue data work with the Project Intersection team.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Begin design of the Library Subscription data.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continued Query tool/web tool investigation.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continued to work on the analysis of the SAP/HR implementation in=
the warehouse.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continued design and implementation of new HR Org hierarchy.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continue work with the Project Intersection team.<br>
<br>
<br>
<b>Next Month's Plans (October, 2002)<br>
</b><font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continued work on Freewilly setup and work on Oracle 8.1.7=
migration. <br>
<font face=3D"Courier New,=
Courier"><x-tab> </x-tab>o</=
font> Continue working on Pro*C program to generate index scripts with=
new <br>
<x-tab> </x-tab><x-tab> =
</x-tab>tablespace names and=
correct new 'initial,'next' parameters. Need <br>
<x-tab> </x-tab><x-tab> =
</x-tab>to develop part of the=
process that will generate these scripts for ALL tables for a particular=
user. <font face=3D"Courier New,=
Courier">o<x-tab> </x-tab>o</font> Identify all components to be=
moved (all conversions involved, all jobs and all scripts). <br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continue work on migrating warehouse Web/CGI applications from=
SNOWDROP to BLUE-GOOSE.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Fully productionalize Graduate Admissions (add metadata to web=
page).<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Load student tuition assessment data.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continue working on moving complete Library piece to Freewilly.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Prepare for AAUDE board meeting in Washington DC.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continue hiring process (Replace Steve Turner who is transitioning=
out of DW as of June 30).<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continue to work on the analysis of the SAP/HR implementation in the=
warehouse.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continue design and implementation of new HR Org hierarchy.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continue data modeling sessions with MIT Medical.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continue data modeling sessions with TLO.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Begin data modeling sessions with the MIT Card Office<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continue data work with the Project Intersection team.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Begin design of the Library Subscription data.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Continued Query tool/web tool investigation.<br>
<br>
<br>
<b>Next Community Milestones:<br>
</b><font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Migration to new server and upgrade database from Orcal 7.3.4 to=
8.1.7 -- November 2002<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Rollout of the SAP/HR/CYBORG - Early 2003<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Load summary (by service) MIT Medical data into the DW.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Load TLO case data into the DW.<br>
<font=
face=3D"Symbol">=B7<x-tab> </x-tab=
></font>Load MIT Card Office data (including pictures) into the DW.<br>
<br>
<br>
</html>
--=====================_133805612==_.ALT--
