Showing posts with label Project. Show all posts
Showing posts with label Project. Show all posts

Saturday, December 10, 2011

Closing all re-eval requests and inquiries.

Received a re-eval request for phase 3 and this was my response. Am posting here because some of it is general in nature and bears wider dissemination.

Hi D,


I reviewed your Team's submission. Before I go into some detail, let me clarify a couple of things.

1. In each of the 10 grading criteria, a "1" means the submission was "at par" or met expectations. For any given grading characteristic, the majority of groups would have scored a 1. So getting a "1" does not mean you did badly. A "1.5" means the submission was 'above par' on that criterion and a 0.5 means it was 'below par' on that criterion. A "2" (very rare and subjective) means the submission was exceptional on that criterion. Hence, you will see that 1.5 was the maximum score on most criteria because some team or the other, on a relative basis would have done well on some criterion or the other.

2. This year, almost all teams did well. The mean was higher and the variance lower than in past years. I suspect it has partly to do with the example PPTs being put out. People were able to learn from past mistakes better and submit better work than average. But that also meant that the par value went up for the "at par" and "above par" ratings.

Your team did well on most parameters. You were at par on many criteria. The 0.5 you got for ROs was because the ROs were unclear. The DPs seemed uni-dimensional (I didn't cut marks here though) and somewhat narrow given the project scope. However the 1.5 in insights obtained is because, despite being narrow, the ROs seemed to have been fully addressed.

Teams that put in some form of animation so that the sequence of steps comes by itself got a 1.5 on creativity. Groups that were able to integrate their analysis into a clean, reasonable set of recommendations did well in 'results obtained' and 'insights' criteria. And so on.

So overall, I'm sorry to say I do not see scope to change the marks as given. Its not because you did badly but because most other teams did well also and the bar was raised for scoring at 'above par' levels.

I hope that clarifies.

I shall close re-val requests for Phase 3 and its time to release the grades soon.

Sudhir


Saturday, November 26, 2011

Eleventh hour Phase III Q&A

Update:
Got this email:
Hi Professor,


Will it be possible to extend the deadline for submission of group project from 6 PM to midnight i.e. 12 AM.

There are couple of reasons behind this request :

1) We followed the instructions for extension of SPSS recommended by IT services. However it involved Virtual Box creation which is a terribly slow way to operate heavy data on SPSS (each excel file is 3-4 MB, and it takes 10 minutes just to transfer data from excel to SPSS run on Virtual Box)

2) A lot of time was spent looking for secondary data which we did not expect.

3) Conflicting deadlines for Pricing, CSOT, ENDM and BVFS between today and tomorrow.

Will be grateful if the extension is allowed.
Regards,
G 
My response:
Hi G and Team,


I have no problem with extending the deadline another 6 hrs.

I don't know how to program turnitin.com.

Chandana has set the deadline for 6 PM on the dropbox.

Let me try to reach her and see if she can extend this to 12 midnight.

Shall inform the class soon regarding this.

Sudhir
Hope that clarifies. Shall let you folks know soon via (yet another) mass e-mail..

Sudhir
------------------------------------------------
Hi all,

Got this series of mails from "t" just now. Have edited to remove specifics and focus on general take-aways to share with the class.

Hi Prof,
One question on cluster analysis-
While choosing variables for cluster analysis, can we pick 3 factor scores which are essentially 3 buckets of multiple variables and two individual variables (ie scores on ‘I am generally budget conscious’ and ‘ I carefully plan my finances’)? Does that distort the output? Can it be interpreted?
Regards,
T
My response:
Hi T,
No, that's perfectly fine. In fact, it is recommended when certain variables don;t load very well onto the factor solution in the upstream stage.
As for interpretation, yes it follows the same as it would if they'd been factors. Factor scores are now essentially variable values where we are concerned in downstream analysis.
And then T replied:
Thanks for that prof. The only problem is that the variables under question are becoming disproportionately important in the 'predictor importance' scores and are pulling down the importance of other factor scores.
At which I wrote:
Make sure all the input variables into a cluster analysis program are standardized (i.e. subtract mean and divide by std dev) before cluster analysis to remove variable scaling effects on clustering. That should help.
Sudhir

P.S.
Will keep you posted as more Q&A happens.

Phase III - Grading Criteria

Hi All,

Might as well outline some thoughts on the grading criteria for the project. These are indicative only and are not exhaustive. However, they give a fairly good idea of what you can expect. Pls ensure your deliverable doesn't lack substance in these broad areas.

1. Quality of the D.P.(s) - How well it aligns with and addresses the business problem vaguely outlined in the project scope document; How well can it be resolved given the data at hand. Etc.

2. Quality of the R.O.s - How well defined and specific the R.O.s are in general; How well the R.O.s cover and address the D.P.s; How well they map onto specific analysis tools; How well they lead to specific recomemndations made to the client in the end. Etc.

3. Quality and rigor of Data cleaning - The thinking that went into the data cleaning exercise; the logic behind the way you went about it; the ways adopted to minimize throwing out useful observations using imputations, for instance; the final size of the clean dataset that you ended up with for indepth analysis. The data section should contain these details, ideally.

4. Clarity, focus and purpose in the Methodology -  Flows from the D.P. and the R.O.s. Why you chose this particular series of analysis steps in your methodology and not some alternative. The methodlogy section would be a subset of a full fledged research design, essentially. The emphasis should be on simplicity, brevity and logical flow.

5. Quality of Assumptions made - Assumptions should be reasonable and clearly stated in different steps. Was there opportunity for any validation of assumptions downstream, any reality checks done to see if things are fine?

6. Quality of results obtained - the actual analysis performed and the results obtained. What problems were encountered and how did you circumvent them. How useful are the results? If they're not very useful, how did you transform them post-analysis into something more relevant and useable.

7. Quality of insight obtained, recommendations made - How all that you did so far is finally integrated into a coherent whole to yield data-backed recommendations that are clear, actionable, specific to the problem at hand and likely to significantly impact the decisions downstream. How well the original D.P. is now 'resolved'.

8. Quality of learnings noted - Post-facto, what generic learnings and take-aways from the project emerged. More specifically, "what would you do differently in questionnaire design, in data collection and in data analysis to get a better outcome?".

9. Completeness of submission - Was sufficient info provided to track back what you actually did, if required - preferably in the main slides, else in the appendices? For instances, were Q no.s provided for the inputs to a factor analysis or cluster analysis exercise?  Were links to appendix tables present in the main slides? Etc.

10. Creativity, story and flow - Was the submission reader-friendly? Does a 'story' come through in an interconnection between one slide and the next? Were important points highlighted, cluttered slides animated in sequence, callouts and other tools used to emphasize important points in particular slides and so on.

OK. Thats quite a lot already, I guess.

Sudhir

Thursday, November 24, 2011

Phase III - D.P. definition travails

Update:
Hi all,

Another Q I got asked recently which, I think, bears wider dissemination.

1. The 40 slide limit is the upper bound. Feel free to have a lower #slides in your deliverable. No problem.

2. The amount of work Phase III may take, by my estimate would be about 15-20 person hours in all. Say about 3 hours per group member. Anything much more than that and perhaps you are going about it the wrong way. The project is the high point of applied learning in MKTR_125. So yes, 3-4 hours of effort on the project is not an unfair amount of load. Besides, effort correlates well with learning, in my experience.

3. Do allow yourself to have fun with the project - its not meant to be some sooper-serious burden, oh no. Keeping a light disposition, a witty touch, a sense of optimism and the big-picture in mind helps with flexibility, creativity, out-of-box and all those nice things, in my experience. Whats more, if you enjoyed doing the project, rest assured it *will* show in the output.

Hopefully that allays some concerns about workload expectations etc. relating to Phase III.

Sudhir

------------------------------------------------------------------------------
Folks,

The example PPTs putup don't have a D.P. explicitly written down because I had given the D.P. in project 2009. In Project 2011 however, you have been given the flexibility to come up with a D.P. of your own, consistent with the project scope. This is both a challenge and an opportunity.

I got some Qs on whether the handset section responses can be ignored because client is primarily a service provider. Well, the handset section contains important information about emerging trends in the consumer's mindspace, the mobile application space and the mobile-related brands' perception space. A telecom service provider would dearly like to know how many target segment people have/are switching to smartphones, which apps they use most often so that the carrier can emphasize those apps more and so on. So yes, the handset section should not be ignored, IMHO.

Hope that clarified.

Sudhir

--------------------------------------------------------------
Got an email outlining whether such-and-such was a good D.P. - R.O. combo.

Now, I can;t share what the D.P. itself was but my response to that team had generalities which might do with some dissemination within the class.

Hi R and team,


Looks good. However, I don't quite see what the decision problem is. Step 1 seems to state more an R.O. than a D.P.

Sure, sometimes a D.P. maps exactly onto a single R.O. and that may well be the case here. But such a D.P. would perhaps be overly narrow considering the project scope initially outlined.

I'd rather you state a broader D.P. and break it down into 3 R.O.s each corresponding to the S, T and P parts of S-T-P.

Well, that's my opinion, you don't necessarily have to buy into it. What you do is ultimately your call.

Do email with queries as they arise.

Good luck with the project and happy learning.
Sudhir
More miscell Q&A:
Dear Sir,


While running the factor analysis on the pychographic questions, do we re-label the levels of agreement and disagreement as 1-5? Will this help us in any way?

Another way out could be to label the responses as 1 and 0 where 1 is for a level of agreement and 0 for disagreement. In the Ice-Cream Survey HW the responses to these psychographic questions were binary which actually helped in the analysis.

A quick response will be really helpful. Our approach is to first segment the consumers using factor analysis and then find some interesting insights into usage based on these segments.

Cheers,

S
My response:


Hi S,
1-5 (or 1-7 in case of a 7 point scale) is the conventional practice. Safe to go with conventional practice.

You might as well choose to go with something else but may need to justify why.

Alternately, copy the data columns of interest onto a new sheet, and reduce the #responses to three (-1 for disagree, 0 for neither and +1 for agree), re-run factor analysis and see if the variance explained, factor structure etc that you now get is better than with 1-5.

Idea is that as and when you approach specific problems, you think up of neat, creative ways to negotiate it and move on. Therein lies the learning in the analysis portion for MKTR. :)

Hope that helped. Do write in with more queries should they arise.
----------------------------------------------------------------------------
Dear Sir,

We are facing an issue with regard to clubbing the dataset with Q48 (in terms of Rank) into the final data set that was uploaded earlier. The following are the discrepancies:

• The new Data set for Q48 does not have serial no’s to link it with earlier set

• The no of rows (entries) in the new data set for Q48 are more than the original one

• There should be ranks from 1 to 3 but we see that some data entries up to rank 9

It would be beneficial for all the groups to have a final data set with Q48(in terms of ranks) extracted from Qualtrics.

Looking forward for a quick response from your end.

Regards,

M

My response:

Hi M,

The respondent ID that is there (leftmost column in the Q48 dataset) can be used to match (via VLOOKUP func in excel) with the same rows in the original dataset. The blogpost made a mention of this specifically.

Once you match the Q48 dataset entries to the original dataset ones, the problem regarding #responses less or more in Q48 dataset also goes away.

The ranks 1 to 3 alone are relevant. Some people ranked more than the top 3 items, so you may get upto 9 but that we can ignore, if need be.

Hope that clarifies.
Sudhir
----------------------------------------------------------------
Received this email today fro group "J":
Dear Sir,

We have the following analysis for our MKTR project:

Decision problem: Foreign handset player wants to enter the Indian market
Research Object: Wants to identify the most lucrative segment to enter.

This would make some of the questions about service provision irrelevant. Do you think we are on the right track?
Regards,
My response:
Hi Team J,


The project scope was designed from the viewpoint of a large Indian telecom service provider.
So a purely handset-maker's perspective would be limiting, I feel and perhaps not entirely consistent with the original project scope document.

My suggestion is you consider modifying the D.P. such that the data on service provider characteristics can also be integrated and used in some manner. For instance, a handset maker looking to ally with a service provider, perhaps.

In general, the D.P. should not be so limiting as to banish a good part of the data we have collected from analysis. There is plenty of scope to creatively come up with D.P.s that while focussing on the handset side of the story also use telecom carrier data.

Hope that clarifies.
Sudhir

Monday, November 21, 2011

Project Phase III Q&A

Update:
A more recent email with similar Qs:

Hi Professor,

Hope you’re well!

I’m writing to ask some help with the data cleaning of the Project’s Phase III data. Can you please help with the following queries?

1. What does it mean when the data field has both blanks as well as “-99” as the response? Thought it was the same thing, yet Q45, for instance, has both fields.

2. Can you please also advise on how to handle coding of ranking/rating questions? Specifically:
a. Q41 – Rating of top two categories of channels watched
b. Q48 – Ranking of only Top 3 brands out of a list of about 10

3. Also, how do we code blank responses (especially for interval or ratio scaled questions)?
a. Do we put a “0” or leave it blank?
b. How does SPSS handle zeroes vs. blanks?

Please advise.
My response:

Hi A and Team,
1. "-99" means respondent has seen the Q but chosen to ignore it. Blank means the respondent never saw the Q, i.e. the skip logic didn't lead him/her to the Q in the first place.


2. Q41 - select multiple options is what was done for the tv channel Q. So, folks have selected 2 (and some have selected more than 2). There is no ranking implicit in what was chosen.

Q48 - I'll download this Q afresh so that the rankings are visible. At present, under 'download as labels', we are unable to see the rankings.

3. a. Depends on how many blanks are there. If the entire row is blank, drop that row. The Q was not relevant to the respondent and hence qualtrics skipped those Qs for him/her. If a few columns here and there are at "-99", then either impite mean/median for that column or "0" for "do not know". Its a call you have to take given the context the Q arises in.

b. Doesn't handle them very well but allows you to do some basic ops. SPSS will ask whether you want to exclude cases (i.e. rows) with missing observations or whether you would rather replace the missing cells with the column means. Choose wisely and proceed.

Am sending the Q48 ranking data afresh to the AAs for an LMS upload. Should be up for viewing and download soon. Use the respondent ID to vlookup and match like rows in the master dataset you are currently working on.

Hope that clarified things somewhat.
Sudhir
-------------------------------------------------------------------------------------------------
Got this email from a team:

Dear Chandana,
Phase 3 project requires a 40 slide PPT as our deliverable. I would like to get the following things clarified:
a)      Is 40 the minimum or maximum limit. It seems too big a task.
b)      Can we generate output tables of tools such as Cluster and Factor analysis and include it as per the content of the PPT or is it that the tables be part of the appendix alone.
c)       Is secondary data analysis mandatory/allowed/is optional. Since the entire survey cannot cover a limit of 40 slides through a primary survey alone.
Kindly help in getting these points clarified as we are in the process of finalizing our approach.

My response:

Hi Team G,
1. The 40 slide limit is the upper limit. Feel free to have your final PPT deliverable less than 40 slides long.
2. There is little point in pasting SPSS output tables for factor.cluster analyses inside the 40 slide limit, IMHO. I'd rather groups present the interpretation/info/insight that emerges from such techniques. A hyperlink to the appendices that contain the SPSS tables shouldn't hurt at all, though.
3. Secondary data usage is welcome as long as the sources are documented and cited meticuluously.
IMHO, the main challenges arise in deciding upon a suitable D.P. and its constituent R.O.s. What follows is straightforward once these are set. I'd say, don't be overly ambitious in defining your D.P. nor overly shallow in scope either.
I hope that clarified things at least somewhat. Pls feel free to write in with queries as and when.

Regards,

Sudhir



Sunday, November 20, 2011

Data cleaning tips for Project Phase III

Hi all,

As you by now well know, one of the exam Questions related to output from your phase III project (the factor analysis one). While analyzing data for this particular problem, I did notice quite a few irregulartities with the data. This is typical. And hence, the need for a reasonable cleaning of the data prior to analysis.

For instance in the Qs related measuring to importance of telecom service provider attributes, there were a few pranksters who rated everything "Never heard of it." Clearly, its stretch to think internet savvy folks have never heard of even one of a telecom carrier's service attributes. Such rows should be cleaned (i.e. either removed from analysis or if there are gaps etc, then imputing missing values for these gaps etc) before analysis can proceed.

Some advice for speeding up the cleaning process:

1. Do *not* attempt to clean data in all the columns. That is way too much. Only for important Qs, on which you will do quantitaive analysis using advanced tools (factor/cluster analysis, MDS, regressions etc) should you consider cleaning the data. So choose carefully the columns to clean data in. Needless to say this will depend on what decision problems you have identified for resolution.

2.  An easy way to start the cleaning process is to check whether there is reasonable variation in responses for each respondent. Thus, for instance, after the psychographic Q columns, insert a new column and in it, for each row, compute the standard deviation of the responses of the psychographic Qs. The respondents with very high or low standard deviations should be investigated for possible data cleaning. Thus for example, this method would catch people who mark the same response for every Q. Or those who mark only extremes for some reason.

3. If there are gaps or missing values (these are depicted with -99 in the dataset) in critical columns, then you may consider replacing these with the median or the mean of that column. A fancier, more general name for such work is imputation which includes also various model based predictions to use as the replacement value. Without imputation, one will be forced to drop the entire row for want of a few missing values here and there. Always ensure, the imputed values are 'reasonable' before going down this path.

More apropriately, imputation might work better if used segment-wise. Suppose you've segmented the consumer base based on some clustering variables. Now since those segments are assumed homogenous in some sense, one can better impute missing values with segment means/medians perhaps.

4. Don't spend too much time on data cleaning also. In the first (half-)hour odd spent on data cleaning, chances are you will cover the majority of troubled rows. After that there is a diminishing returns pattern. So draw a line somewhere, stop and start analysis from that point on.

Update:
I'll share my answers here on the blog to any Qs asked to me by groups. Conversely, I request groups to first check the blog for whether their Qs have already been answered here insome form or the other.

Hope that clarifies.

Sudhir

P.S.
Any feedback on the exam, project or any other course feature etc is welcome. As always.


Friday, November 18, 2011

Homework and Exam related Q&A

Post-Exam Update:
I later noticed, after the exam had started that there was a typo in the binary logit question. The coeff of income^2 was shown as -.088 instead of -0.88. Hence, the predicted probabilities of channel watched for all 3 cases (min, max and mean profile of respondents) would now come up as 1. Folks who show the expression and calculations will get full credit for this problem.

A related Q regarding the DTH caselet. There are many possible assumptions you could make and from each, a different research design might flow. As long as you've stated clearly your assumptions and the research design that follows is logically consistent with your assumptions, you are OK.

Hope that clarifies.

Sudhir

Update:

Another thing I might as well clarify: - regarding the quant portion - only interpretation of a model and its associated tables will be asked for.

Even there, only those tables that we explicitly have discussed in class, not merely shown but discussed, will be important from an exam viewpoint.

Sudhir

Am getting quite a few Qs regarding this. A wrote in just now:
Hi Chandana,

Need a quick clarification. For this home work questions 1,2,3 and 4 are mandatory and 5,6,7 are optional. Is that right ?
Regards,
A
My response:
Sudhir
----------------------------------------------------------------
Got this just now:
Professor,

When I mailed you today morning, I had high aspirations of finishing my studies and then meeting you for a quick review. Unfortunately, I just finished going through Session 4 hand out. My sincere apologies but I guess I will have to cancel this appointment…and give the slot to a better prepared student…!!

Thanks,
K
My response:
That's OK. Drop by anyway. I expected I'd be busy during these office hrs but am (pleasantly) surprised. Seems folks have on average understood the material well and don't need additional office hrs now.

Besides, anyway, the exam is not designed to be troublesome. I wouldn't worry overmuch if I were you.

Sudhir
----------------------------------------------------------------


Hi all,

Was asked yesterday about this and might as well share with the whole class.

Q was that in the factor analysis homework, what to do if the factor solution with eigenvals>1 is still showing cumulative variance explained of <60%?

Well, if the % is in the mid to late 50s, just go with it.

If not, it would seem like the factor solution is not doing a great job of explaining variance in the input variables. This is presumably because at least some variables are not well correlated with the others and are hence weakening the factor solution.

To ID such variables, look either at the correlation tables or at the communlaities table. The variables that show least correlation with others should progressively be removed and factor anbalysis re-conducted at each step till the 60% criterion is met.

If a variable loads entirely on its own factor, drop that factor and use that variable as-is in downstream analysis.

Hope that clarifies. Any more such Qs and I shall share it here.

Sudhir
Hi A,
Yes, only Q1-4 are mandatory. Rest are optional in session 9 HW.

Thursday, November 17, 2011

Project related Q&A

Update:

I split up the old post which had both Q&A as well as deliverable format. The old post is now exclusively deliverable format and is available here:

http://marketing-yogi.blogspot.com/2011/11/more-phase-iii-project-q.html

A recent post on take-aways from past projects (particularly those in 2010) can be found here:

http://marketing-yogi.blogspot.com/2011/11/take-aways-from-past-projects.html
------------------------------------------------------------------------------

Hi all,

Got asked a few Qs, might as well putup answers here.

1."The decision problem is vague. Should we focus more on telecom services or handset features?"

My response: The scope document is vague for good reason. Provides enough wriggle room to variously interpret decision problems. Pls come up with your own decision problem that is not-inconsistent with the project scope.

Regarding telecom service versus handset, well, the scope doc says the client XYZ is a major player in the telecom services space - so it is primarily a service provider. The handset features thing is likely secondary to its main goal of improving its position among telecom service providers.

Pls come up with a suitable D.P. and appropriate downstream analysis. The main theme of interest will be, how much does your D.P. align with the scope document and how logical and consitent is the donwstream analysis with the stated D.P.

2. Scope says, XYZ wants to know "where the market is going". What does this mean?

Well, XYZ will want to know many things, sure, but only so many are knowable with the data at hand. My intention behind that part was, illuminate the current standing of different offerings first and then project a few years into the future. Make assumptions as necessary. There'll be little data to back up projections into the future, understandably so. Shall later expand on wht I mean by this point.

More Q&A as they come will be on this page as updates.

Sudhir
-----------------------------------------------------------

Update: More project related guidance

In general, there are a few Questions which the client is likely to find of interest and which I think each project team could consider going through.

(i) What is the turnover (i.e. rate of change or switching) among attractive customer segments in telecom carriers and handsets? [Hint: See how long one has been with a carrier/handset Q among others]. Is there a trend? Anything systematic that might indicate the market is moving towards or appears to be favoring a certain set of attributes more than others?

(ii) What kind of usage patterns, apps etc are the attractive segments (say in terms of purchasing power or WTP etc) moving towards? More voice? More data? More something? From here can flow recommendations to the client on which applications to focus on, which platforms to explore alliances with etc perhaps.

(iii) Perceptual maps of current reality - where attractive customer segments perceive current carriers and/or handsets to be. What are the critical dimensions or axes along which such percpetions have formed? What are the attractive gaps in positioning that may emerge? Which niche perhaps can a new or existing service lay claim to on the positioning map?

More Qs will be added as they occur.

Pls note that it is *not* necessary that these Qs be answered or recommendations made along their lines. Its just a guide for teams to think about incorporating into their current plan/roadmap. Incorporating these Qs makes sense only if they align with the D.P.s and the R.O.s you have chosen.

Sudhir 

Wednesday, November 16, 2011

Phase III Project - Deliverables and format

Hi Class,


Your deliverable consists of a 40 slide PPT (excluding appendices). Should be emailed to me with a copy to the AAs before deadline: 27-Nov 6 pm.

More specifically, your PPT should contain:


1. Filename (mandatory) - should be of the form group_name.pptx when submitted.

2. Title slide (mandatory) - Project title - something concise but informative that describes the gist of what you've done. In addition, the title slide should also contain your Group Name, Team-member names, PGIDs  MKTR section.
Pls note, make no mention of your team member names anywhere else in the PPT, have it only in the title slide.

3. Presentation Outline (Optional) - a Contents page that outlines your presentation structured along sections (e.g. methodology, [...] , recommendations, etc).

4. Decision Problem(s) (Mandatory) - State as clearly as feasible the decision problems (D.P.s) you are analyzing Give numbers to the D.P.s if there're more than one.

If small enough in number, you may also list the R.O.s you have in this slide.


5. A Methodology Section (Mandatory) - In preferably a graph or flowchart form, lay down what analysis procedures you used, in what order to answer particular Research Objectives (R.O.s) that cover the D.P.s.

6.A Data section (Mandatory) - explains that nature and structure of the data that were used. Be very brief but very informative - write (ii) the dimensions of the data matrices used as input to in different procedures, and (ii) the sources of data - cited sources if secondary data are used, and Question numbers in the survey questionnaire if primary data are used.

I strongly suggest using a tabular format here. Packs a lot of info into compact space. Easy to read and compare too.

Some clue as to what filters or conditions were used to clean the data would be very valuable also.


7. Model Expressions (Mandatory)- Write the conceptual and/or mathematical expressions of any dependence models used. Then directly use the results in downstream analysis.

Kindly place in the appendix section a descriptives table of the input data, a brief explanation of the X variables used, and of course, output tables along with interpretation.

8. Appendix Section (Optional): Some of the less important tables can be plugged into a separate appendix section (outside the 40 slide limit) in case you are running out of slide space. Have only the most important results tables in the main portion.

9. Recommendations (Mandatory) - crisp, clear, in simple words directed towards the client. Emphasize the usability and actionability of the recommendations.

Further, I strongly advise groups to make their PPT deliverables reader-friendly. That is:
- use as simple language as feasible.
- Animate the slides if they are cluttered so that when I go through them later, the proper sequence of material will show up.
- Highlight keywords and important phrases.
- Use callouts etc as required to call attention to particularly important points.
- An economy of words (sentence fragments for eaxmple) is always welcome.

I hope that clarifies a lot of issues with deliverable format. Groups that flout these norms may lose a few points here and there.

Sudhir

Sunday, November 13, 2011

Take-aways from past projects

Update: Received a few such emails so might as well clarify:

Hi Prof,

There seems to be some confusion on the final report submission due date. This is to check whether submission is due in session 10.

Kindly confirm.
S

My response:

No. Its due on 27-Nov. - the day before term 6.
Hope that clarifies.
Sudhir
----------------------------------------------------------
Hi all,

As promised, some collected thoughts on where folks ran into obstacles in past years. Am collating and putting up relevant excerpts from past blogposts to start with. Shall add and expand this thread as more info comes in.
------------------------------------------------------------------
From last year (financial planning and savings avenues analysis)


[Some] stuff that IMHO mertits wider dissemination.
1. Let me rush to clarify that no great detail is expected in the supply side Q.
As in, you're not expected to say - "XYZ should offer an FD [fixed deposit] with a two year minimum lock-in offering 8.75% p.a.".
No.
Saying "XYZ should offer FDs in its product portfolio." is sufficient.


2. Make the assumption that the sample adequately represents the target population - of young, urban, upwardly mobile professionals. 

3. Yes, data cleaning is a long, messy process. But it is worthwhile since once it's done, the rest of the analyses follow through very easily indeed, in seconds. 

4. It helps to start with some idea or set of conjectures about a set of product classes and a set of potential target segments in mind, perhaps. One can then use statistical analyses to either confirm or disprove particular hypotheses about them.

5. There is no 'right or wrong' approach to the problem. There is however a logical, coherent and data-driven approach to making actionable recommendations versus one that is not. I'll be looking for logical errors, coherency issues, unsustainable assumptions and the like in your journey to the recommendations you make in phase III.
------------------------------------------------------------------
From last year on stumbles in the analysis phase:
1. Have some basic roadmap in mind before you start: This is important else you risk getting lost in the data and all the analyses that are now possible. There are literally millions of ways in which a dataset that size can be sliced and diced. Groups that had no broad, big-picture idea of where they want to go with the analysis inevitably run into problems.

Now don't get me wrong, this is not to pre-judge or straitjacket your perspective or anything - the initial plan you have in mind doesn't restrict your options. It can and should be changed and improvised as the analysis proceeds.

Update: OK. Some may ask - can we get a more specific example? Here is what I had in mind when I was thinking broad, basic plan from an example I outlined in the comments section to a post below:
E.g. - First we clean data out for missing values in Qs 7,10,27 etc -> then do factor analysis on psychogr and demogr -> then did cluster analysis on the factors -> then we estimate segment sizes thus obtained -> then we look up supply side options -> arrive at recommendations.

Hope that clarifies.
2. Segmentation is the key: The Project essentially, at its core, boils down to an STP or Segmentation-Targeting-Positioning exercise. And it is the Segmentation part which is crucial to getting the TP parts right. What inputs to have for the segmentation part, what clustering bases to use, how many clusters to get out via k-means, how best to characterize those clusters and how to decide which among them is best/most attractive are, IMHO, the real tricky questions in the project.

3. Kindly ask around for software tool gyan: A good number of folk I have met seemed to have basic confusion regarding factor and cluster analyses and how to run these on the software tool. This after I thought I'd done a good job going step-by-step over the procedure in class and interpreting the results. Kindly ask around for clarifications etc on the JMP implementation of these procedures. The textbook contains good overviews about the conceptual aspects of these methods.

I'm hopeful that at least a few folk in each group have a handle on these critical procedures - factor and cluster. I shall, for completeness sake, again go through them quickly tomorrow in class.

4. The 80-20 rule applies very much so in data cleaning:Chances are under 20% of the columns in the dataset will yield over 80% of its usable information content. So don't waste time cleaning data (i.e. removing missing values, nonsense answers etc) from all the columns, just the important ones only. Again, you need to have some basic plan in mind before you can ID the important columns.

Also, not all data cleaning need mean dropping rows. In some instances, missing values can perhaps be safely imputed using column means or medians or the mode (depending on data type). 

Chalo, enough for now. More as updates occur.
Sudhir
------------------------------------------------------------------
More from last year on specific problems encountered in final presentations:


1. Research Objective (R.O.) matters.
Recall from lectures 1, 2 & 3 my repeated exhortations that "A clear cut R.O. that starts with an action verb defined over a crisp actionable object sets the agenda for all that follows". Well that wasn't all blah-blah blah. Its effects are measurable, as I came to see.

Suppose the entire group was on board with and agreed upon a single, well-defined R.O., then planning, delegation and recombining different modules into a whole would have been much simplified. Coherence matters much in a project this complex and with coordination issues of the kind you must've faced. It was likely to visibly impact the quality of the outcome, and IMHO, it did.

2. Two broad approaches emerged - Asset First and Customer First.
One, where you define your research objective (R.O.) as "Identify the most attractive asset class." and the other, "Identify the most attractive customer segment." The two R.O.s lead to 2 very different downstream paths.

Most groups preferred the first (asset first) route. Here, the game was to ID the most attractive asset classes using size, monetary value as addressable market or some such criterion and then filter in only those respondents who showed some interest in the selected asset classes. Then characterize the indirect respondent segments obtained and build recommendations on that basis.

I was trying to nudge people towards the second, "Customer segmentation first" route partly because it aligns much more closely with the core Marketing STP (Segmentation-Targeting-Positioning) way. In this approach, the entire respondent base is first segmented along psychographic- behavioral - motivational or demographic bases, then different segments are evaluated for attractiveness based on some criterion - monetary value, count or share etc, and then the most attractive segments are profiled/analyzed for asset class preferences and investments.

Am happy to say that in a majority of the groups, once a group implicitly chose a particular R.O., the approach that followed was logically consistent with the R.O. 

3. Some novel, surprising things.
Just reeling off a few quick ones that do come to my mind.

One, how do you select the "most attractive" segment or asset class given a set of options? Some groups went with a simple countcriterion - count the # of respondents corresponding to that cluster and pick the largest one. Some groups went further and used a value criterion - multiply the count with (%savings times average income times % asset class allocation) to arrive at a rupee figure. This latter approach is more rigorous and objective, IMHO. There were only 2 groups that went even further in their choice of a attractiveness criterion - the customer lifetime value (CLV) criterion. They multiplied the rupee value per annum per respondent with a (cleaned up) "years to retirement" variable to obtain the revenue stream value of a respondent over his/her pre-retirement lifetime. Post-retirement, people become net consumers and not net savers, so post-retirement is a clean break from pre-retirement. I thought this last approach was simply brilliant. Wow. And even within the two groups that did use this idea, one went further and normalized cluster lifetime earnings by cluster size giving a crisp comparison benchmark.

Two, how to select the basis variables for a good clustering solution? Regardless of which approach you took, a good segmenting solution in which clusters are clear, distinct, sizeable and actionable would be required. One clear thing that emerged across multiple groups was that using only the Q27 psychographics and the Demographics wasn't yielding a good clustering solution. The very first few runs (<1 minute each on JMP and I'm told several minutes on MEXL) should have signaled that things were off with this approach. Adding more variables would have been key. Typically, groups adding savings motivation variables, Q7 constant sum etc were able to see a better clustering solution. There is seldom any ideal clustering solution and that's a valuable learning when dealing with real data (unlike the made-up data of classroom examples).

One group that stood out in the second point approach used all 113 variables in the dataset in a factor analysis -> got some 43 odd factors -> labeled and IDed them -> then selectively chose 40 from among the 43 as a segmenting basis and obtained a neat clustering solution. The reason this approach stands out in my mind 'brute force approach' is that there's no place for subjective judgment, no chance that some correlations among disparate variables will have been overlooked etc. It's also risky as such attempts are fraught with multi-collinearity and inference issues. Anyway, it seemed to have worked.
[...]
Anyway, like I have repeatedly mentioned - effort often correlates positively with learning. So I'm hoping your effort on this project did translate into enduring learning regarding research design, data work, modeling,  project planning, delegation and coordination among other things.
------------------------------------------------------------------
OK, that's it for now. Shall update the thread with more learnings from past years for reference purposes.


Sudhir

Thursday, November 10, 2011

Project Updates

Update: Received a few such emails so might as well clarify:
Hi Prof,


There seems to be some confusion on the final report submission due date. This is to check whether submission is due in session 10.

Kindly confirm.
S
My response:
No. Its due on 27-Nov. - the day before term 6. 
Hope that clarifies.
Sudhir

Update:
The Phase III dataset is available for download from LMS. Happy analyzing.

'-99' is the code for a question seen but not answered. The proportion of -99s for different Qs might give some clue as to the cost of not forcing those Qs to be answered.

Some exemplery projects have also been putup on LMS from MKTR 2009 car survey. Teams Ajmer, Mohali and Kargil did well whereas Jhumri-Taliya didn't do so great.

Just got done with the general tutorial/Q&A in AC2. My thanks to those who showed up. Got some valuable feedback as well. Some possible to implement right away for session 9 & 10, perhaps.

I'll make detailed slides on how-to on SPSS so that getting too much into the tool does not occupy class time. You can go home and practice the classroom examples at leisure. I'll focus more on analysis results and their interpretation. Unlike many other courses you may have taken, MKTR is a tool-heavy course, so some level of engagement with the SPSS tool is unavoidable.

Hope that clarifies.

Sudhir
-------------------------------------------------------------------
Hi All,

Am closing the survey responses for MKTR Project 2011 - we have some 3,153 responses in all. After getting rid of say about 20-25% invalid responses, we may still have well above 2000 responses. Quite sufficient for most of our purposes, I reckon.

I'll putup a preliminarily cleaned version of the dataset in Excel file format for upload on LMS by noon 11-Nov Friday. Phase III can then begin. Pls also find some exemplary PPTs on the MKTR car project 2009 analyses on LMS by Friday noon.

Sudhir

Wednesday, November 9, 2011

9-Nov Phase II interim results

Hi all,

We've some 2600+ responses. Am hoping for 2000+ valid responses now. Great going!. Here's the latest status-check on which team's where.

Row Labels Count of V1


Agra 73

Bhilai 77

Bijapur 60

Cochin 59

Dehradun 67

Dhanbad 37

Dum dum 119

Durgapur 125

Gulmarg 80

Guntur 76

HasmatPet 50

Jaipur 83

Jamshedpur 81

Jaunpur 71

Jhujuni 254

Kakinada 67

Kesaria 107

Naihati 88

Palgat 112

Panaji 55

Peepli 51

Portblair 44

Rampur 117

Sangli 73

Shivneri 55

Sultanpur 82

Team Leh 47

Trawadi 61

Trichy 61

Ujjain 27

(blank) 294

Grand Total 2654

Teams with 8-10 respondents per team member, (or, say, 60+ valid responses per team) are at par. So don't worry overmuch about it.
 
Sudhir

Monday, November 7, 2011

7-Nov Phase II interim status-check

Hi All,

Am extremely pleased to say we've close to 1700+ responses already. Assuming even a quarter of them are unusable, we'll still have 1000+ responses to play around with. Good! And still we have some 3 days to go. Great!

Here's the latest status-check for where groups are currently:

Agra 57


Bhilai 58

Bijapur 40

Cochin 35

Dehradun 51

Dhanbad 29

Dum dum 105

Durgapur 108

Gulmarg 30

Guntur 53

HasmatPet 27

Jaipur 44

Jamshedpur 48

Jaunpur 65

Jhujuni 160

Kakinada 18

Kesaria 81

Naihati 30

Palgat 81

Panaji 27

Peepli 27

Portblair 30

Rampur 61

Sangli 65

Shivneri 49

Sultanpur 67

Team Leh 29

Trawadi 13

Trichy 50

Ujjain 20

(blank) 204

Grand Total 1763


I'd say about 8-10 valid responses per team member would be par for this phase. Of course, more is merrier. At or around par would earn most groups 5% of the total 8% grade. The top quarter of groups get 7 or maybe 8% based on the actual numbers.

Soon, very soon, we'll be up and running on this one with Phase III.

Sudhir

Saturday, November 5, 2011

Where groups stand currently in Phase II

Hi all,


Qualtrics is reporting some 784 responses so far. IMHO, that is quite encouraging indeed. Linear extrapolations are dicey but at this rate I can cautiously hope we'll meet the 1500 target on time.

Of course, of these 784, about a fifth odd would be invalid/incomplete responses, on the average (in the optimistic case). Some surveys take less than a minute, it seems and others report opening-closing of several hours. There's been murmur about people taking the surveys more than once on behlaf of the same or different groups. Will have to check IP addresses and weed out duplicates for this.

Am attaching below the status thus far of what I received.

Group #responses


Agra 14

Bhilai 34

Bijipur 15

Cochin 12

Dehradun 26

Dhanbad 9

Dum dum 54

Durgapur 69

Gulmarg 20

Guntur 18

HasmatPet 9

Jaipur 17

Jaunpur 44

Jhujuni 36

Kakinada 2

Kesaria 34

Naihati 10

Palgat 54

Panaji 7

Peepli 6

Rampur 40

Sangli 47

Shivneri 43

Sultanpur 27

Team Leh 14

Trawadi 1

Trichy 26

Ujjain 6

Blank 89

Will update again on Tuesday. Shall putup the names column of folks who have responded on LMS on Monday (I don't quite know how to operate LMS, will need to wait for the AAs to come on Monday and do this ) so that groups can check if their contacts have responded or not and follow-up accordingly.
 
The number of blanks (at 89, its easily the mode) is worrisome. Respondents aren't taking the time and trouble, apparently to record the group's name that sent them the survey. Pls reinforce the importance of this aspect amongst your contacts, folks.
 
Phase II is truly underway and soon, very soon, there'll be plenty of data to play around with. Plenty of it. And Phase III will be upon us. The division of grade is roughly 8%, 8% and 19% for the 3 phases, respectively.
 
Pls use the comments section of this post for clarifications or other Q&A, as far as possible.
 
Sudhir

Wednesday, November 2, 2011

Project Phase II is here

Hi All,

Happy to inform that we've somehow managed to put together a reasonably concise questionnaire (49 odd questions in the longest string) based loosely on the project scope document. Sure, some aspects had to be left out as they would have made the survey unweildy but that was a call one has to make at different points in time in MKTR.

Here is the phase II final survey link:


Shall ask the AAs to putup the corresponding word doc on LMS.

Should you find any bloopers in the questionnaire, pls let me know ASAP.

Some points to note:
1. Pls email your contacts the above link to take the survey. Phase II ends 10-Nov Thursday noon. Responses coming in after this will not be counted for analysis.

2. Will release the consolidated database of responses for analysis to the class by Friday 11-Nov noon. Phase III will begin in earnst then

3. Pls call-up, SMS, gently-remind your contacts off-campus to fill-up the survey. In particular, the last section has a question asking respondents to select from a drop-down the name of the group that sent them the survey. Hence, Pls ensure the respondents know your group's name! We'll simply tally the group responses and the groups bringing in the most valid, completed responses score higher in Phase II. This weekend should be a busy time and am hoping we get a good many responses then.

4. Pls ensure your group shows up for the guest lecture tomorrow. Attendance there will be worth between 2 and 3 % of the course grade, just FYI. Even now I see the sign-up sheet is half-empty.

Regards,
Sudhir

Saturday, October 29, 2011

Project Discussions - Open Thread

Update: Evening Saturday 29-oct
Sir,
There are some demographics questions that I was planning to use from the previous surveys (not reinventing the wheel). Will that be a problem for turnitin?
Regards
M

My response:
Hi M,

We're not using turnitin to check similarities across submissions. Its just that LMS doesn;t seem to have an electronic dropbox facility. Go ahead and borrow from external sources as appropriate. No problem.

Sudhir


Hi All,

Shall compile project related Q&A here. Pls feel free to shoot away.

Meanwhile, spied a rather pertinent piece in the Economic Times yesterday that seems quite relevant to the project and also to some of the topics we've covered in MKTR - notably demand estimation & market sizing.

The Age of low cost

The idea is that low cost computing and connectivity devices have come of age both in terms of installed base and in terms of critical support to break through the chicken-and-egg cycle of "enough-apps to excite market development  versus enough market to excite apps development". I strongly recommend a thorough read from the project and also the course POV.

Here are some nice excerpts:
It took three decades for computing devices in India to reach the 50 million installed mark. The next 50 million will take a tenth of that time. Even then,India with a base of 50 million personal computers now would lag behind China (300 million) and the US (394 million).There is still plenty for room to grow.

Hmmm. Think of the structural breaks and trend upheavels in the sales curve from 1980 to 2015. The growth will likely slow and plateu now but still...

Yet, he's visibly excited about the 1,750 tablet,recently launched by the government and the slew of new computing devices Asus,Samsung,HCL,Reliance and others have rolled out in the last two months.Almost all of them are available around the 10,000 price point."They are good for a web based lifestyle,"he says.Sonali Garg,19,a Chandigarh based Commerce student who shares a laptop with two other siblings is also eyeing these new devices."We can buy tablets with saved up pocket money,"she says.At far away Agartala,20-year-old Bishnesh Das fancies them too."These gadgets will do to computing what sub 3,000 phones did to mobile communications,"he says.
The app side is no less exciting. From obvious uses in the education space to less obvious ones like agriculture. Here are some teasers...

Netbooks for text books - that's the switch technology training company NIIT has done in the last six months for all students in its premier GNIIT course.In the next six months it will migrate all its courseware onto netbooks and tablets."Students won't carry books,but tablets to class,"says Vijay Thadani,CEO,NIIT. [...]
S Sivakumar,chief executive,agri business,says crop management advice can be personalized to individual farmers,if they can video or photo shoot the field conditions and transmit to experts via tablets."Through use of video/photo transmission,price negotiation process can be instant and more effective.Order aggregation for farm inputs will help in streamlining logistics and reduce costs,"he adds.Mass adoption,says Tuli of Datawind,will happen when such devices help users generate business."Today a phone is a commerce tool for all segments of users.Same will happen with computers.When mobile phones hit the market no one thought your neighbourhood small merchant or a rickshaw puller will buy them.Today they all do."
More on how costs have fallen in real terms (the supply curve shifts rightwards bigtime) and the stellar effect that's having on sales and thereby, demand:

More applications and content will make low-cost computing more relevant to consumers.But cost is crucial too.Many expect Mukesh Ambani to be a game changer.ET recently reported that Reliance Industries will unveil a new range of 4G-enabled tablets at 3,000.Such a price point would have been ridiculously low even one year ago.But prices of components like the hard disk,RAM etc have come down enabling manufacturers to come out with innovative offerings.Says NIIT's Thadani: "The real breakthrough has come from Moore's Law: processor power doubles every 18 months and costs come down.There's more power packed in each new generation of computers."Alok Bharadwaj,the president of,Manufacturers Association of Information Technology (MAIT) attributes low cost computers to economies of scale and an average decline of 15% a year in component prices.Adds Apratim Sharma,country product manager,Asus India: "We observe 15-20 % yearly drop in cost of same hardware'.You can get a hard disk that went into high end laptops 2-3 years back at lower costs in netbooks or tablets now,he adds.Asus just launched a 10 inch net book at 9,999 with a 250 GB hard disk,1GB RAM,web cam,WiFi and Bluetooth."Three years ago a similar device would have cost double,"he adds.NIIT's Thadani still remembers the first PC they bought back in the 1990s.It cost 1.5 lakh.It had 10 MB hard disk and 5 MB RAM."Now,you get a far more richer device for less than 10,000,"he adds.

That's enough for now I guess. Read it all. Shall add updates in chronological order at the top of this thread (more recent on top).

Sudhir

Thursday, October 20, 2011

Admin Issues and Updates (Post Session 2)

Hi All,

First off, welcome to the course to all final registrants. The project group formation exercise must kickoff ASAP and complete by midnight 23-24 Oct.

UPDATE 1: Quite a few groups have used group names that aren't consistint with the group naming policy. Pls use names of Indian small towns only, folks. Anything from Kargil to Kanyakumari is fine.

UPDATE 2: Section C's names weren't putp last time. That's been rectified in the google spreadhseet now.

Pls find below a link to a googe doc where you can write your group name next to that of yours and of your group-mates.

https://docs.google.com/spreadsheet/ccc?key=0AiXdfyuoSD-ZdHB1VlNRaUFKR1F6eFJVRDBsX24wTEE&hl=en_US

Pls elect one person as contat-person for your team. The contact person should email the AAs the list of members in his/her team (names, PGIDs, sections) and copy all the team members, for the record.

There are two small McKinsey reports (about a dozen pages each) uploaded on LMS, following session 2 and which may be useful in your homeworks.

Pls go through the Project scope document below and let me know if anything is unclear, missing, can be easily improved etc. Pls feel free to use the comments space in the blog for feedback.

I'll later also put up a poll for session 2 after Section C's class is done (and likewise for every session thereafter) to collect feedback about each session. I'd appreciate your constructive critique and inputs.

Pls feel free to email me any queries, comments or suggestions.

Thanks and regards,

Sudhir

Wednesday, October 19, 2011

2011 Project Scope Document (Frozen, Final Version)

Context: XYZ Corp, a large domestic conglomerate is contemplating revamping its Indian telecom carrier business. With the advent of rapid technological advances both in the hardware (Tablets, iProducts) as well as, concurrently, on the application side - 3G, GPS, gaming, MMS etc., the market is in flux.

The idea is to divine out where the market is going in terms of trends, customer tastes and preferences etc and thereby pre-position XYZ to take advantage of these trends and movements in the near future in the form of strategic stakes in incipient firms, alliances with established upstream or downstream players, re-positioning and re-branding exercises and so on.

XYZ Corp is currently engaged in a descriptive study of the structure and competition levels in the India market on the one hand and the preferences, usage patterns, habits, attitudes and perceptions of current and prospective Indian customers on the other.

You are a team of B2C India experts within a large, global management consultancy. XYZ Corp seeks your inputs and insights on the customer side of its study.

Scope: XYZ wants to know (among other things, perhaps) about the habits and preferences of current and prospective customers that can be used to forecast the shape and direction of the mobile carrier market for the next few years.

A few indicative examples are listed below (not necessarily in any order of importance):

1. Mobile Usage patterns among consumers. These may include:

(i)the intensity of mobile use (For example - how many minutes per day, local or long distance, personal or work related etc),

(ii) mode of use (E.g., Voice/Text/Data/Web) etc. Work or personal etc.

(iii) Average spends on different modes.

(iv) ease and comfort with m-commerce or its older cousin e-commerce

2. Mobile handset and carrier particulars. These may include:

(a) Which handset? (For example -Basic phone, smartphone, tablet, iPhone) etc. Which kind of service - prepaid, postpaid? Etc

(b) Which OS? Which applications most commonly accessed and used? Which games most popular?

(c) Preferences over particular handset - which features are considered essential, which nice to have and which are considered unimportant.

(d) Features or quality of service afforded by the telecom carrier

(e) some hint of what might delight customers - what apps would the customer like to see, etc.

3. Perceptions and Loyalty for current and prospective customers.

(i) How often are handsets changed? How often are carriers changed?

(ii) Which handset brands rated the best? On which dimensions?

(iii) Which telecom carriers rated highest? On which dimensions?

(iv) expectations of current and prospective customers

(v) feedback from current customers

4. Demographic and lifestyle information.
(i) size of social network, comfort with social media,
(ii) who influences purchase decisions in general - peers, opinion-leaders etc.
5. Etc.

Please note, the above are indicative only. Groups are expected to come up with a better list of their own.


Research the web for listings of handset features, technical specs, and carrier evaluation attributes, past research, secondary data etc. in the construction of your questionnaire.

Requirement: (1) Design a questionnaire for a web-based survey that can be administered to people in the target segment. The survey shouldn't take the typical respondent more than 20 minutes to answer. Please leverage the strengths of a web-survey - notably the ability to imbed IF-THEN loops within the questionnaire.

Hence for example, in a survey on car attribute preferences, you could have a question that asks "Are you: (A) currently a car-owner (B) in the market to buy a car, (C) not looking to buy a car" and depending on whether the respondent checks (A), (B) or (C), irrelevant questions are skipped and only the relevant ones presented.

First, brainstorm and come up with a questionnaire draft on a regular word editor. Make sure it has these IF-THEN loops readied. Refine the draft and ready it for submission. Questionnaire design will be covered in session 3. Get a heads up with reading the summary portions of the relevant textbook chapter on the same. You DONT have to reinvent the wheel. So please feel free to look up readymade past questionnaires on the web. Cite sources if you took help. However, no outright plagiarism please!

(2) Please make a web-survey. Register for an ISB account on qualtrics (email ITCS for help on this one). I will also putup instructions on the blog. Use qualtrics tutorials etc to understand how to create your survey with the logical loops built in.

It is recommended to test the web-survey you have built by having your project group members take it and point out ambiguities, inconsistencies, skip-logic that isn't working etc. Do NOT administer the survey to anyone outside MKTR yet.

(3) E-mail the academic associate both (a) the link for the web-survey and (b) the word copy of the questionnaire. Please remember to mention group name and members list in both the web-survey and the offline questionnaire.

The deadline for the submission of survey is 12 noon on Sunday - 30 October.
 
(4) I and the AAs will provide you with a *final* version of a questionnaire which the entire class will then administer using the ISB account for qualtrics. That will come later and will be detailed at an appropriate time.

UPDATE:
Had put this project scope doc first up on the blog. Pls find released the full, formal and final version of this document above.

Hope that helps. Questions and comments welcome.

Sudhir.

Sunday, December 5, 2010

Grading related mailbag

Two emails I received (made anonymous) and my responses therein. IMHO, bear wider dissemination:

On peer eval impact on project grading:

    Can you please let us know the weight given on peer evaluation out of the total weight of 40 marks on the group project.


    Regards
    R

My response:

    Its a maximum of 4 out of the 40 for the project.


    And it is given only if there is concurrence (2 or more people in the group have peer rated a deviation similarly). We only had a handful of cases where the criteria were met.


    Sudhir

On phase II marks allocation:

    Apoorva,


    First thing I said I am fine with the marks for phase III.


    I just rechecked the standings on the number of responses which was put up on the professor’s blog finally.

    We were placed X th out of 37 teams.

    So, we were higher than median even without considering the ISB responses and also that we were a smaller team of 6.

    Now, it was communicated earlier to us that we will be given credit only for non-ISB responses and also depending on the team size.

    So, I do not understand why we should be getting a score of 7 which is below average. Clearly we performed better than more than ½ the teams so we would expect to get well above average.

    And as the prof. has pointed himself in his blog post there is not a big variation in any component other than this we cannot afford to get less on this when we deserve credit.

    Regards,
    V

My response:

    Let me answer this one.

    1.       ISB responses were considered except where IP addresses were clearly duplicated. The large numbers of foreign and otherwise recently nonresident students in several teams would have been disadvantaged otherwise. However, ISB responses were discouraged for others and rightly so. In any case we had about 10% of the total responses from within campus – not a magnitude that would reverse the current rank order or anything.

    2.       Groups that made the minimum threshold – 10 valid responses per team member – got the baseline score of 7.  This is the vast majority of teams – 26 out of the 36 we had (Apoorva, correct me if I’m mistaken here).

    3.       The top 5 teams in the total responses sweepstakes got additional credit of 1.5. Their totals were far higher than average responses/team, the responses seemed genuine and regardless of whether or not one normalizes by team size, their output was high and thereby merited additional credit – so I deemed.

    4.       The bottom 5 teams got about 0.5 points less than the baseline 7, depending on the actuals.

    5.       Over all the average was 7.18 and you stand where you do.


    Now, while I make no claims that this particular grading scheme was perfect or anything, it is consistent and objective. IMHO, of course. Changing anything at this stage will kickoff changes for groups above yours as well as those par –the vast majority of groups there are – and will likely nullify any grade related gains one may hope to make thereby. Hence, I shall go with what is currently done.

Sorry abt the delay in letter grades release. Shall be done soon.


Sudhir

Friday, November 12, 2010

Project presentations learnings summary.

Class,

As I write this, am done with all group presentations. Would rather pen down learnings early on before memory and other priorities take their toll.

First off, given the time and other constraints (you had less than a week between dataset-access and deliverable presentation), the output was by and large commendable. Some quick general observations:



1. Research Objective (R.O.) matters.
Recall from lectures 1, 2 & 3 my repeated exhortations that "A clear cut R.O. that starts with an action verb defined over a crisp actionable object sets the agenda for all that follows". Well that wasn't all blah-blah blah. Its effects are measurable, as I came to see.

Suppose the entire group was on board with and agreed upon a single, well-defined R.O., then planning, delegation and recombining different modules into a whole would have been much simplified. Coherence matters much in a project this complex and with coordination issues of the kind you must've faced. It was likely to visibly impact the quality of the outcome, and IMHO, it did.

2. Two broad approaches emerged - Asset First and Customer First.
One, where you define your research objective (R.O.) as "Identify the most attractive asset class." and the other, "Identify the most attractive customer segment." The two R.O.s lead to 2 very different downstream paths.

Most groups preferred the first (asset first) route. Here, the game was to ID the most attractive asset classes using size, monetary value as addressable market or some such criterion and then filter in only those respondents who showed some interest in the selected asset classes. Then characterize the indirect respondent segments obtained and build recommendations on that basis.

I was trying to nudge people towards the second, "Customer segmentation first" route partly because it aligns much more closely with the core Marketing STP (Segmentation-Targeting-Positioning) way. In this approach, the entire respondent base is first segmented along psychographic- behavioral - motivational or demographic bases, then different segments are evaluated for attractiveness based on some criterion - monetary value, count or share etc, and then the most attractive segments are profiled/analyzed for asset class preferences and investments.

Am happy to say that in a majority of the groups, once a group implicitly chose a particular R.O., the approach that followed was logically consistent with the R.O.

3. Some novel, surprising things.
Just reeling off a few quick ones that do come to my mind.

One, how do you select the "most attractive" segment or asset class given a set of options? Some groups went with a simple count criterion - count the # of respondents corresponding to that cluster and pick the largest one. Some groups went further and used a value criterion - multiply the count with (%savings times average income times % asset class allocation) to arrive at a rupee figure. This latter approach is more rigorous and objective, IMHO. There were only 2 groups that went even further in their choice of a attractiveness criterion - the customer lifetime value (CLV) criterion. They multiplied the rupee value per annum per respondent with a (cleaned up) "years to retirement" variable to obtain the revenue stream value of a respondent over his/her pre-retirement lifetime. Post-retirement, people become net consumers and not net savers, so post-retirement is a clean break from pre-retirement. I thought this last approach was simply brilliant. Wow. And even within the two groups that did use this idea, one went further and normalized cluster lifetime earnings by cluster size giving a crisp comparison benchmark.

Two, how to select the basis variables for a good clustering solution? Regardless of which approach you took, a good segmenting solution in which clusters are clear, distinct, sizeable and actionable would be required. One clear thing that emerged across multiple groups was that using only the Q27 psychographics and the Demographics wasn't yielding a good clustering solution. The very first few runs (<1 minute each on JMP and I'm told several minutes on MEXL) should have signaled that things were off with this approach. Adding more variables would have been key. Typically, groups adding savings motivation variables, Q7 constant sum etc were able to see a better clustering solution. There is seldom any ideal clustering solution and that's a valuable learning when dealing with real data (unlike the made-up data of classroom examples).

One group that stood out in the second point approach used all 113 variables in the dataset in a factor analysis -> got some 43 odd factors -> labeled and IDed them -> then selectively chose 40 from among the 43 as a segmenting basis and obtained a neat clustering solution. The reason this approach stands out in my mind 'brute force approach' is that there's no place for subjective judgment, no chance that some correlations among disparate variables will have been overlooked etc. It's also risky as such attempts are fraught with multi-collinearity and inference issues. Anyway, it seemed to have worked.

Three, when entry into real estate financial products, a regional break up that plays upon the geographic variation captured in the primary data would have been a good thing from the XYZ point of view. Would help them prioritize and better allocate resources in managerial decision making etc.

Anyway, like I have repeatedly mentioned - effort often correlates positively with learning. So I'm hoping your effort on this project did translate into enduring learning regarding research design, data work, modeling,  project planning, delegation and coordination among other things.

I wish you all the best in your future endeavors.

Goodbye and Goodluck.

Sudhir

P.S.
Would appreciate constructive feedback that may help the MKTR course next year. Many thanks to the kindred souls in Section D for the 'Thank You' card, I thought that was sweet.