Standard Jet DBnb` Ugr@?~1y0̝cßFNa7ޜY(,`{6j߱nCP53y[ /|*"|r,mf_Љ$g'DeFx -bT4.0Ydv YS  Y   Y Y  Y Y  Y  Y  Y   Y  Y  Y  Y  Y 2Y  Y   Y  Y ConnectDatabaseDateCreateDateUpdate FlagsForeignNameIdLvLvExtraLvModule LvPropName OwnerParentIdRmtInfoLongRmtInfoShortTypeYYIdParentIdName        OY@SY Y Y  Y 2ACMFInheritableObjectIdSID  AtYObjectId Y SEY  Y Y Y  Y  Y Y  Y AttributeExpressionFlagLvExtra Name1 Name2ObjectId Ordernzf edY"ObjectIdAttribute -YSY Y Y  Y  Y  Y  Y  Y ccolumn grbiticolumnszColumnszObject$szReferencedColumn$szReferencedObjectszRelationship   YYYszObject$szReferencedObjectszRelationshipYv1b N  : k & W  C t/  @@ @   kQJiMWSdi`ifmiQqYQsfJbQ^Md``QbmkOJmJJMMQkkfJUQkOJmJLJkQkSdi`k `dOo^QkiQ^JmYdbkWYfkiQfdimk kMiYfmk kvkiQ^mJL^Qk6:+hkQ^+iQqYQsfJbQ^iQfdimkQJiMW`kvkJMMQkkdL[QMmk`kvkJMQk`kvkdL[QMmk`kvkhoQiYQk`kvkiQ^JmYdbkWYfkmL^Mm`JMMQkk^Jvdom`kvkOLko``JivYbSd okQiOQSYbQO Z  @ @ @          @ @ @ @ @ @ @ @ !"#$%&'()*+,-./0123789 : ; < @ A B  456   9  @ 88B8<8=8>8?8@8A8C 8D888 8 8 8 8 8 8 8   d _ Z 7 T  _  g  ɑ  ?S@S@AccessLayout4MR2KeepLocal T"@zz:::::::8 @ NS@NS@rptReviewPanelComments@ ZZZNNNNNNNL @ -l}S@a}S@SearchForm@BBB66666664 @-l}S@S@02_qSel_ReviewPanelReportSearch, `Y@|@:xxl``````^ ` H|S@{eS@tblCTM@:::......., @0S@0S@DataAccessPages@@@@@@@@@@> 0S@0S@SysRel.........., 0S@0S@Modules0000000000. 0S@0S@Scripts0000000000. 0S@0S@Reports0000000000. 0S@0S@Forms,,,,,,,,,,*  0S@0S@UserDefinedA@DDD88888886 @ 0S@0S@SummaryInfol@DDD88888886 @0S@0S@MSysAccessObjectsDDDDDDDDDDB =S@=S@MSysRelationships}DDDDDDDDDDB =S@=S@MSysQueries}88888888886 =S@=S@MSysACEs}22222222220 =S@=S@MSysObjects}88888888886 =S@=S@MSysDb@:::......., @=S@=S@Relationships}<<<<<<<<<<: =S@=S@Databases}44444444442 =S@=S@Tables}.........., C{hUB/A S @ ,  |  B /   l Y 2 o\I"r_K8   RR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np      RR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np    RR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np   RR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np  z  RR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np  z  RR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np  z  RR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np  z  RR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np  z  RR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np  z  RR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np     RR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np   RR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np  RR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np   RR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np  RR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np TRR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np TRR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np RR:tn?M&B[YZhBV nUoMm?1E\CA6iFGQҦi-_Sb[Np     z  z     +LVAL ДSSyyyyyyyyyyyyyyyyyyyyyyyyyMR28DescriptionGUIDNameMapL~ vPopup form -- openMR28DescriptionGUIDNameMap~ vPopup form -- openMR28DescriptionGUIDNameMap~ vPopup form -- opens listbox to select review panel comments .kd*Gpu6 . U MR28DescriptionGUIDNameMap~ vPopup form -- opens listbox to select review panel comments .kd*Gpu6 . U MR2RecordLocksODBCTimeoutMaxRecordsRecordsetType FilterOrderByOrderByOnOrientationDefaultViewReplicableNameMapGUIDDOLS  <     ⣞&L jmaRw  U{S?_CaZ=DPS@tblCTM{lNJY\{S?_CaZ=DTrackNoIDpkg@ƒAh+—{S?_CaZ=DBinFinalReportraFg{S?MR28DescriptionGUIDNameMap~ vPopup form -- opens listbox to select review panel comments .kd*Gpu6 . U MR2ANSI Query Mode(Themed Form ControlsAccessVersion BuildProjVerN    08.50 . #MR2( ReplicateProject MR20 Title AuthorCompany8 db1 Thomas Bock Y NY Y (DataIDDDjp(YAOIndexv1 u  @ @;<=>? s r   K[p;rȚccessObjectsDDDDDDDDDDB K[ࡱ> Root EntrybPropData Forms Reports#080:<=Babc$ghiXj,k"[Event Procedure]rS@4qIN=N~qb U⣞&L jmaRwS@02_qSel_ReviewPanelReportSearch{lNJY\⣞&L jmaRwTrackNoIDpkg@ƒAh+—⣞&L jmaRwBinFinalReportraFg⣞&L jmaRwBinHandbook&D>⣞&L jmaRwBinOBEPd-BHa.o⣞&Root Entry?bPropData Forms Reports#L jmaRwBinPAR nACvX⣞&L jmaRwBinRequiresClarificationBuDK$5G7$I4qIN=N~qTrackNoIDpk >02_qSel_ReviewPanelReportSearch>02_qSel_ReviewPanelReportSearchVerdana8$h !@m߀rvd  25 .#3)+,;=( /1<GHIJLOMNyRTUVWXYZ[\]wxz{|SDataAccessPages Databases 0 p)Blob   !"5 932%* $-046/7;:=(<GHIJLOMNyRTUVWXYZ[\]wK[xz{|Sdir(50`D)?Blob BMTypeInfoCj__SRP_1 \__SRP_4&$__SRP_5pBlobDeltazPropData5PropData:DirData;dir(50`D)Blob BKTypeInfoCjModulesScriptsVBA@CmdbarsCustomGroups DirDataVBAProjectp)@AcessVBAData hm5C Arialn= Arialo3A Arialp4`ReportHeader04+DS_` Detail@(L+.m;F`<ceTrackNoIDpkTrackNoIDpk:Primary key referencing issueBuDK$5G7$Ij`K[iBinFinalReportBinFinalReportχOg MGӟޖwj`? iBinHandbookBinHandbookrQ@c Mz Nj`i BinOBE BinOBEJ.[M__SRP_1 \__SRP_4&$__SRP_5p6 1 !"#$%&'2)*+,-./01234<8>?A@IJDEFGH}KLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|~entsjdjdjdjdkVBA\VBA6\VBE6.DLL#Visual Basic F 028=Babctghij&kd{Zͬ_"iE9JUDetailh cmdGojchkFinalReportd Label6jchkHandbookd Label8jchkOBEd Label10j chkPARd  Label12j chkRequiresClarificationd  Label14k fraAndORd Label16iOption18d LaijMSysDbID="{15C015C4-E420-43FB-B211-4CA1CC50494D}" DocClass=Form_SearchForm/&H00000000 DocClass=Report_rptReviewPanelComments/&H00000000 Name="db1" HelpContextID="0" VersionCompatible32="393222000" CMG="37358C7E94822686268626862686" DPB="C5C77E0C829B839B839B" GC="5351E8827783778388" [Host Extender Info] &H00000001bel19iOption20d Label21on20d Label21or Applications*\G{4AFFC9@.kd*Gpu. U NSelect Values from Both//Either ListboxVerdana8 h !@m߀rvdBplB d5`a8bcLabel21Orݷ~I.@_,!ortBinFinalReportχOg MGӟޖdhijknhjdjdjdjdjdkdididAR}kGGw=Gj`ͬ_"iE9JUDetailh cmdGojchkFinalReportd Label6jchkHandbookd Label8jchkOBEd Label10j chkPARd 8} MY,4N ,,,,Y Y  Y  Y  Y Y TrackNoIDpkBinFinalReportBinHandbook BinOBE BinPAR0BinRequiresClarificationjjj 3jjjj 4jYY EopIDPrimaryKey (Y  Y  $TrackNoIDpk"ClonedTrackNoIDpkPARIDfkCTMCTMSublevel SourceWorkingGroup$CollectionAnalysisObservationDiscussionRecommendationBinFinalReportBinHandbook BinOBE BinPAR0BinRequiresClarificationBinCommentDateCreatedUserCreatedDateModifiedUserModifiedReportDateAppendDate 3 4YY EopIDPrimaryKey  DateCreated EopIDPrimaryKey  teCreated EopIDPrimaryKeyUserCreated  teCreated1 EopIDPrimaryKeyUserCreated  Created1 EopIDPrimaryKeyUserCreated  vvvvvvvvvv1  LVALʾtblCTM02_qSel_ReviewPanelReportSearchSearchFormrptReviewPanelComments^3LVAL L < D z|TIWG's primary focus seemed to be standing up the physical networks.CTO covered the environment aspect on one slide. Specifically they noted that the weather will be fixed throughout the test. This eliminates the design of experiment aspect of the CTM. It is necessary to make some assumptions in order to conduct the test within time/money constraints. However at CTM 0 to eliminate factors means that the test could be missing a critical factor.CTO used red and threat during their briefing as well as on their slides.On several slides as well as during the brief CTO referred to step 0.2.4 as Red. This is inconsistent with the CTM methodology and could lead to confusion.Mr. Mike Sutton said that there is a weakness in defining the JOC-T that the CTM needs to address. There is a lack of a single/standard system engineering tool for use in developing the operational context. There is a need for something that identifies the key players and initializes the process for building the logical design. The JBD2 test event groups are also already formulating the logical and physical design concurrently with the JOC-T before the CTM 0 is complete.Should the CTM provide possible sources for information. Where to go to get what? Maybe it becomes part of the handbook?Not clear what can be done in parallel and what is required to be sequential. Product inputs and outputs help but the iterative nature of CTM steps causes confusion.Current UJTLs are more focused on measures for training and not always quantitative that can be evaluated sufficiently in joint testing.What happens if the physical design does not match with logical design requirements? Logical design would indicate what is required for test resources. If the physical design does not match and there is no volunteer to have an asset played, then the CM must determine how to get the resource or determine if it can be done without the resource.LVALR 0 2 ,4|hClarify/come to agreement with CTO on necessary DoDAF Artifacts as inputs/outputs.Recommend the Test Plan as the output for CTM 1.2--perhaps named Initial Test Design.For TE 2, help the OCWG Chair refine his unconstrained JOC-T into a manageable set of systems and operational interactions. For the CTM, recognize that overall Test Goals and Objectives must be identified be a useful JOC-T can be developed and tailored to test the effectivness of a particular system or SoS.Given the critical importance of the CJI for a useful CTM evaluation, it is recommended that the M&P Development Division quickly review this CJI to determine if a CTM 5 evaluation can predictably flow from this wording.Help the OCWG Chair describe the notional JBD2 SoS under test. This does not need to be a description of hardware. It can be a "capability" description based on the system performance criteria extracted from the FCS JCD or ICD or TEMP as appropriate.Create separate data management working group (DMWG).Query intelligence agencies responsible for threat models regarding CTM 0.2.4.CTO follows the CTM and does a full design of experiments in CTM step 0 and then based on weighting (not perfect method, but it would work) of subject matter experts eliminate unimportant factors.Mixing terminology causes confusion. CTO needs to be reminded of the goal to use the CTM and to follow it consistently.Recommend using consistent terminology.Include a process for identifying key players for the logical design. Consider allowing CTM 0 to support initial development of logical and physical design.Decide if this should be included and where to include the information.Need to ensure the JCAs and UJTLs provide the format and information to support testing in a JME. Would also like to see some form of automation that would aid in identifying applicable JCAs and UJTLs.Physical design development needs to be iterative to be tailored to the logical design.+8  b .(IvN@YS@N@`O@0 -- Develop T&E Strategy0.3MPCOCWG1@ @ @@---stafford.maheuTomsqkf`. ':b1M@/?S@M@`O@1 -- Characterize Test1.1OCWG---1v@ @ @---kenneth.levanTomymkf`Z. '&V?QM@ΕS@M@`O@0 -- Develop T&E Strategy0.3IPC---1@ @@Examine for consistency.kenneth.levanTom~rpkf`. %(\OM@6S@M@`O@0 -- Develop T&E Strategy0.3IPC---1$@ .@j@ JOC-T has evolved beyond its original intent.kenneth.levanTom ~rpkf`. $oM@,S@M@`O@0 -- Develop T&E Strategy0.3IPC---1@@@ ---kenneth.levanTom~rpkf`. '# Y M@DDDS@M@`O@0 -- Develop T&E Strategy0.2IPC---1:@@@ ---kenneth.levanTom~rpkf`. '"Kh/L@S@L@L@2 -- Plan Test---MPCTIWG1Z@@j@PAR 2.thomas.bockTomui][UPJ. G rE@S@E@C@1 -- Characterize Test0.2PDRJDWG1B@@@PAR 4.daniel.grenierTomymke`Z. G;L].E@ϊFS@E@C@0 -- Develop T&E Strategy0.2PDR---1@@@Make decision to keep or not in Handbook.maureen.shortTom~rpkf`. z5E@lS@E@C@0 -- Develop T&E Strategy0.3PDR---1CTM Terminology.@@Combine with tracking # 25 as appropriate.maureen.shortTomrpkf`. ֹE@*F S@E@C@0 -- Develop T&E Strategy0.2PDRPDR1Use of CTM Lexicon Terms.8@N@Delete if appropriate.maureen.shortTomrpkf`. :A@SS@A@C@0 -- Develop T&E Strategy---IPC---1!Weakness in defining the JOC-T.@8@Revise recommendation as appropriate.mark.fiebrandtTom#rpkf`. 7?@/S@?@C@0 -- Develop T&E Strategy---------1d@@@---mark.fiebrandtTom~rpkf`. ᐖ?@S@?@C@4 -- Manage Test---------1f@L@Need clarity in the CTM.CTM 3. Illustrate CTM with non-sequential graphic.mark.fiebrandtTom& xl`^YTN. 8&?@_,S@?@C@0 -- Develop T&E Strategy---IPC---1f@@@Input to PAR 1.mark.fiebrandtTom~rpkf`. GKs?@_!S@?@C@1 -- Characterize Test---IPC---1@@@---mark.fiebrandtTomxlje`Z. @LVAL  RLZThe similarities with JOC-T and JME caused confusion by participants as to their meanings and what is included with each.According to the CTM, a T&E evaluation strategy should "ensure the intent is met and serve as the blueprint to achieve the test objectives in the test concept development process." The JBD2 evaluation strategy as briefed did not identify any test objectives.The JOC-T might be expected to describe the overall collection of forces acting jointly and the TTP they would employ to achieve combined mission effects they could not achieve separately. Developing the JOC-T should include an operational assessment to constrain the context into a manageable set of systems and operational interactions that are relevant and useful in achieving test goals and objectives.Following the Concept Development Conference, the OCWG Chair developed the following CJI that he presented at the opening of the IPC: "Is the JBD2 SoS operationally effective and suitable to conduct the full range of military operations as part of a Joint Task Force." This CJI appears to have been developed without adequate consideration of the definition and purpose of a CJI and the measures that may be expected to flow from a properly developed CJI.JBD2 is a notional SoS that is intended to be the "system under test" for this test event. It will be necessary for this SoS to be described and its system performance criteria identified before a test and evaluation strategy can be developed (CTM 0.3.1).The CTM calls for DoDAF artifacts as products for Analyze Threat. Threat models and/or live systems are handled/resourced outside the control or scope of the Capability Portfolio Manager or Program Manager (for that matter the Testers as well). I question the relevance/usefulness of DoDAF descriptions of the threat laydown when threat representations are managed separately.LLVALv `0fClarify intent of the call for tools and infrastructure characterization so early in the CTM process.The CTM calls for the Enterprise JFM to be an input at CTM 1.4.1 Develop Initial LVC-DE Operational Design. Recommend examining whether or not it is useful to indicate it as an input in an earlier process.For the purposes of test event, JTEM M&P should survey this test planner to determine if he did not understand the definition and purpose of the JOC-T, or if he understood but chose not to use it. If the test planner did not understand the JOC-T, recommend that future versions of the CTM clearly describe and outline the component parts of the JOC-T and the utility of the OV-1 graphic for illustrating only the high level operational concept of the JOC-T.JTEM products (Handbook?) will have to include recommended roles and responsibilities for lead range, lead customer, supporting ranges, and secondary customers. Key organizations will have to have assigned go/no go decision authority (i.e. Who says "No"? Who establishes good idea cut off to mitigate requirements creep?) as achieving everyone's test requirements in every test event is probably infeasible. ---Request that TE2 working group leaders track their progress in the CTM and announce to participants at the beginning of each working group meeting. Determine if OCWG turned to MDMP because CTM was found to be unsuitable or inadequate.Help the OCWG Chair refine the evaluation strategy and identify desired operational outcomes, measures of performance, and measures of effectiveness. Finally, help the OCWG Chair develop a coherent, actionable test scenario, vignette(s), test trials, and an executable test trial matrix. For the CTM, we must recognize that some of this confusion arises because we have the Capability Manager developing a T&E Strategy in CTM 0 before the overall test goals and objectives have been developed in CTM 1. This is an obvious flaw and contradiction in our CTM methods and processes.