We are of course aware of the very large amount of work which must have been involved on the part of JST President and staff members both before and during our visit, and are deeply grateful for such a careful time and effort investment. We are however sure that President Kawasaki and his staff have themselves been happy to demonstrate to us the excellence of the work done by their organisation, and that they have derived some justified pride from our unhidden admiration of their accomplishments !
For future similar evaluation visits, it would be useful to plan for the reviewers to spend at least one half-day together after they have done all the visits, to prepare on the spot a joint report, at least in a preliminary form. It might also be useful for JST to appoint ahead of time, from among the reviewers, one Chairperson, who would collect the individual reports and would have the responsibility of making sure they arrive in time and are properly combined. This has been the practice of RIKEN, and has been even improved by the fact that the Chairman of their evaluation committee could come with his efficient secretary who took appropriate notes during discussions. The Chairperson thus entrusted with more work and responsibility should of course be offered an appropriate financial compensation, as well as his/her secretary.
We are now all far away from one another, and quite busy with other matters; therefore, instead of one collective report of the evaluation team, JST will obtain a series of individual reports, certainly much harder to make use of.
Like my colleagues, I shall therefore be compelled to use the first person of the singular ("I") to express my opinions, instead of the "we" that I have used for the first paragraphs of this Report. I have not been in contact with my colleagues about these paragraphs, but I am absolutely sure that they would have agreed with their contents, if not with their specific formulation. For the rest of this Report, I shall however use the "I" which is required by the circunstances.
To ensure comprehensiveness and to facilitate the use of my report, I shall follow the plan used in the "Interim Summary of Evaluation Report on Basic Rsearch Programs" , produced by the domestic reviewers.
I should also mention at the beginning that I have benefited from conversations with Japanese colleagues, before, during and after our evaluation visit, to complement the information obtained during this visit. Also, my previous involvement in an evaluation visit of RIKEN has helped me grasp some of the subtelties of the Japanese system of research. Similarly, all my colleagues had been previously involved, most quite intimately, with Japanese research policy and Japanese scientists. These are certainly prerequisites for the appointment of similar evaluation teams in the future.
| A - JST Programmes and Basic Research in Japan |
| JST basic research programmes account for
only a moderate fraction (14 %) of the total
basic research programmes of JAPAN. I think
this is quite appropriate. The important selective investment resulting
from JST's programmes must continue to be
made possible by the existence of a much
less selective allotment of smaller grants
to many more research teams. Some of these
will thus gain a "first level"
visibility, and thereby become eligible for
a major boost from JST. Such a policy can
be described as "preservation of the humus from which can
grow flowers".I believe there is no magic formula
for the optimal ratio of selective/automatic
allocation of research funds, and the present
distiribution (50 % MEXT, 14% JST, 36 % others,
presumably including RIKEN) is as good as
any. The quality of the research programmes funded by JST is recognized as undoubtedly very high. This is the opinion of the "non-JST" Japanese scientists I have interviewed : they envy of course their JST-supported colleagues, but recognise that they have been very well selected, and make very good use of the extra means they have obtained. I have not registered suspicions of biases, of nepotism, of cliques. This does not mean that un-selected teams are all of lower quality, or would not have done as well in using supplementary funding, but I think JST cannot hope for more than to ensure that all the teams it supports are externally considered to be well chosen. The JST beneficiaries will never be "the best in the country", but JST should hope that "most of them are among the best". It is my opinion that they are. It is absolutely sure that the selective programmes of JST have done much to ensure a much higher visibility for Japan's basic research. We have not had in-depth discussion in Japan on the specific procedures used to select the programmes from among the many applications received. However, the comments I have obtained from non-JST Japanese colleagues were uniformly positive : they do not complainof the complexities of procedures and uncertainties of the outcome, or of the need to prepare long and detailed proposals. They may of course regret not to have been selected for a special boost, but recognise that the teams selected are excellent, and have been chosen by reasonable procedures. One of the domestic reviewers asked quite pointedly whether JST programmes were really "pioneering", and contribute to the overall improvement of basic research level in Japan ? Only the future will give an undisputable answer to that very important question. I have the feeling that this answer will be very positive. I have also the conviction that if the selective programmes of JST had not been imagined and launched, or if the corresponding funds had been distributed in a less selective manner, or in a less streamlined or more automatic or more arguable manner, then certainly nobody would ask the question of a pioneering role : the answer would be obviously negative. |
| B - Strategic Nature of Basic Research Programmes : |
| I confess to have been a little shocked
by the insistence of most of our Japanese
colleagues to "measure" the quality
of the JST individual programmes first and
foremost by the number of papers published
in Science or Nature. To assign to this indicator
a quasi-exclusive role is, in my opinion,
much too submissive, and minimizes the importance
of all the major specialised journals. I
have myself published a few papers in both
journals ; these papers were by far not my
best and most innovative ones, but they were
more in line with the policies of the Editors
of the time of these journals. If this emphasis
during our visit was motivated by the comment
in favor of this indicator (and of Nobel
Prizes !) by one of the domestic reviewers,
then I beg to be allowed to regret it. It
would be nice to be able to use a hard criterion
for the quality of basic research and its
importance ; none is unfortunately available
on a short term. It appears however that, even in Japan, the extension and the quality of the JST programmes could benefit from more exposure to the public. It might be useful, to ensure the persistence of these programmes and of the JST philosophy, to invest in a more efficient public-relations effort, both in Japan and outside. It might be considered useful to organize an "event" that would attract the media, for instance a yearly meeting of the JST beneficiaries, with public lectures and not only scientific ones. It might lead to the development of a team spirit among the JST teaml leaders, and provide some public exposure. One of the domestic reviewers insists on the necessity to prepare data adapted to the requirements of US agencies such as FDA and EPA, in order to develop applications of the results of basic research done in Japan. He recommends not to let Japan be satisfied with being in a "second group" of nations. I think that he has a point there, but that the problem is much beyond reach of JST. Probably MITI, which has played a major role a few decades ago, should be made more reactive now to the necessities of the day, but I do not believe this to be a task for JST. Anyway, probably any country must be satisfied for the time being with remaining in the "second group" of nations, not only in research, but as well in arms development, or in film or music production. Willy, nilly, the US constitute certainly alone the "first group". During our visit, we have witnessed several cases of close physical interactions between industry and research groups (Sony, Matsushita…). I know of several others, which is a welcome change from the years when the only contacts worth mentioning in Japanese Universities were for the preparation of Ph.D.s by Masters seconded to the Universities by Industry. I have been very impressed by the "international" quality of the JST-supported team leaders we have met. They were fluent in english, which did reflect their extensive exposure to the World at large. They were also quite at ease in placing their work in the international context. My own previous extensive contacts in Japan, since the '60s, are in line with this observation : I do not know whether our Japanese colleagues really measure how much the situation has changed. JST benefits from this change, but is also instrumental in reinforcing it. From our own experience in Strasbourg with the JRDC programme (where we had benefited from a pre-ICORP programme), I can testify that the cooperarion thus engendered has run quite beyond our friendly contacts with the Japanese leaders of the associated teams : joint publications continue to come from cooperation with their junior partners. Thesse remarks, and the recommendation of the domestic reviewer concerning what he calls "Publicity" could lead to a recommendation : to set aside a small fund to help Japanese researchers during the last phases, or mostly after completion of their JST-supporrted programmes, to go abroad to give lectures or take part in conferences, to describe what they have achieved with JST's help. |
| C - Relationship between Research and Universities |
| I wish to address first one specific problem
which arose during our discussions about
the implementation of the ERATO and ICORP
programmes. This is related to the request
to the Japanese beneficiaries of these programme
to rent laboratory space outside their University.
I came to Japan convinced this was a bureaucratic
and unjustified requirement, leading to a
dispersion of facilities, to difficulties
of communication, and to the loss of contact
with younger scientists, including students.
After our visits, I have changed opinion, especially after having seen what has been achieved at Keihanna’s Matsushita Research center, and comparing it with the laboratory of the university wevisited (a CREST-supported programme of high quality)-. The facilities in the first case are admirable, the resarch personnel is obviously very happy with the situation and the only problem in my opinion lies with the fate of the extensive equipment at the end of the programme (more about that later). Quite in contrast, the set-up in the Japanese Universities is very much in line with the traditional situation in other Universities, where safety regulations are either non-existent or disregarded, where corridors and offices are disordered, where sound working practices (not to speak of the GLP's enforced in industry) can certainly not be taught. Also, the exposure of the students to an international environment is necessarily at a much lower level, even in a first-class University. I have no doubt this has nothing to do with the quality of the scientific work carried out there, but one should strive at reforming, with JST's help and under JST's pressure, these bad habits, however familiar they are.. I would suggest that JST could make it a strict rule in awarding its help that the receiving University should bring the corresponding space up to international levels of safety and cleanlyness, and maintain it at that level. On the other hand, operations at sites like Keihanna or Tsukuba should be helped to secure students from Universities, to place young people in excellent working conditions, and see what happens ! I mentioned my concern with the fate of heavy (or even of unique) instrumentaiton at the end of the contract. In our JRDC (ICORP) contract, we have observed the absurdity of the explicit rules, but fortunately also the gracious way used by JST to escape the consequences of this absurdity. From the funds available for the contract, we had bought heavy equipment (NMR spectrometer, diffractometer). This was inventoried, and was supposed to be returned to Japan at the end of the contract. Fortunately, it was nevertheless possible for us to retain it (and to modernise it at our expense), whereas if it had had to be shipped back, it would have been a poisonous gift for any Japanese laboratory after the ordeal of the transport. I assume that the theory for Japanese laboratories benefitting from a JST programme is that they should return equipment at the end of the contract. I trust that this clause, which may be required for administrative reasons, is only implemented with flair…. However, one aspect seems not to have been mentioned during our visit, nor in the report of the domestic reviewers : low-tech or medium-tech used equipment if often quite robust, and may well be extremely useful for Universities or research centres in developing countries. For instance, I am sure that crates of used glassware or of simple heating or measuring equipment would be very useful in the Universities of Viet Nam or Laos, or in Africa, while not really appreciated in Japanese Universities (and certainly not in JST-supported research groups). This could be a low-cost (packing + transport) way to improve the role of Japan in the aid to developing countries. The same does not apply to high-tech equipment, as indicated above. . Finally, one of the domestic reviewers mentions a problem we have not been made aware of during our visit : the consequences on teaching of the award of a CREST or ICORP programme, leading to the possible loss of "teaching manpower" by the Universities. My personal position is that a long range research programme should never lead its protagonists away from teaching. Judging from the presentations given to us by the group leaders we have met, most of them (or all..) would make excellent teachers, and could use their research excellence to convey enthousiasm to young students. The objective, in my opinion, should be that Universities make full use of the excellence of JST beneficiaries, not that these are withdrawn from contact with the young students. Of course, the worst possible scenario would be if Professor X, an excellent teacher and a first-class researcher, obtained support from JST, and as a consequence would be forced to withdraw fully from teaching. The uncompensated loss of teaching manpower would lead to an increased load for his less fortunate colleagues, and would necessarily create dangerous animosity against the JST programmes. In the long run, this situation could even lead to the disruption of JST's programmes ! |
| D - Coordination with other Funding Organizations and Sharing of Roles |
| This is a point which we have hardly discussed, as far as I can remember. The domestic reviewers mention it and JST has provided some explanations. I wish to register my full support for the policy described, of regular meetings of the various agencies to compare their programmes, and presumably to harmonize them. |
| A - ERATO |
| The major point (apart from off-campus/on-campus
operaetions, already discussed and well analyzed
by the domestic reviewers) is certainly the
problem of the Five-Year Research Period. I concur with the domestic reviewers to believe that a strict time limit should be set up : what has not been achieved in five years of well funded research might well be terminated anyway. However, this is one of the points which we had collectively discussed during our visit : the problem recognized by JST is a real one : a 5 year period leads often to 3 1/2 years of actual research, due to the initial lag time (setting up of facilities, buying equipment, screening of candidate researchers…). We (in this case certainly "we", not only "I" !) welcome the explicit indication that JST will study a more flexible management system allowing researchers to use a full 5 year duration, and even an "extension of some projects in the future on the basis of interim evaluation results". I also express my great satisfaction at the large freedom extended to the research directors. I wished our administrations would become as far-sighted ! |
| B - ICORP |
| From our experience in Strasbourg, I know how smotthly and generously these programmes run, and also how useful they are for all parties, in terms of ressources, of intimate cooperation, and in terms of long range interactions (much longer range than the duration of the programme). I think the proposal made by JST in the Interim Summary of Evaluation is excellent, to link up ERATO and CREST programmes with subsequent ICORP programmes. This would have the disadvantage that in some cases it might be preferable to fund an international programme between groups of similar financial prosperity, whereas a CREST-supported group would in many caes be much richer that the foreign group, and this might lead to difficulties. However much I would be glad if we could again benefit from being partners in such an ICORP programme, I think the policy of striving for a global distribution of programmes, and in particular to launch one for the Pacific region, is an excellent one. The problem raised by one of the domestic reviewers, of the management of intellectual property rights, is a very touchy one. It would call for a concerted pressure from interested parties against the US practices. |
| C - CREST |
| The CREST programmes have been absolutely essential for the development in Japanese Universities of some major centres of excellence. As I said earlier, despite the imbalance thus caused between beneficiaries and non-beneficiaries, these programmes appear to be very highly praised by our Japanese colleagues. It is possible that the invitation of a few foreign scientists to take part in the selection process would improve the international visibility of these programmes, and also improve their acceptance by the Japanese scientific community. It is also certainly important to make sure that the selection of a research group by JST to become a CREST beneficiary does not lead to unexpected expenses by the corresponding University, as suggested by one of the domestic reviewers, and as amply demonstrated in some other countries. This would not only be detrimental to the image of the programme and to the acceptance of the beneficiaries by their peer group, but might even well lead to such harsh criticisms that it could endanger continuation of the CRSET programme ! |
| D - PRESTO |
| We (again, "we" !) have been very impressed by the
personal quality of the PRESTO beneficiaries
we have met, and by the interest of the work
they have presented. I consider this programme as extremely important, as it may not only help young researchers to start efficiently in their carreers, but also it is a most useful tool to disrupt the traditional pyramidal University system of the Koza, which seems to be criticised by all my Japanese friends, even by those who are benefitting from it ! Of course, this is not without danger : as one of the domestic reviewers say, "the relationship between reaearchers and their senior personnel with their home organisations could well be soured and undermined". This is also reflected by the explanation given by JST. This programme cannot be jeopardised by such problems, and JST cannot push blindly excellent young researchers into predictable difficulties. I have no clear suggestion to make in this connection. One cannot hope simply to convince senior colleagues that their fame will be greater if they are generously supporting their junior partners. Maybe they should be given some recognition (why not financially, at a modest level ?) to thank them for having taught and nurtured such brilliant successors ? Or, more cheaply, JST should perhaps create some special title, for instance in inviting these distinguished Professors to attend once a year a meeting where their former students, now PRESTO beneficiaries, would present their work, and in bestowing them a "PRESTO Father" certificate ??? Attention should also be drawn to the remark of one of the domestic reviewers, that in some (most ?) cases, the young researcher adds the name of his senior Professor to his papers as a matter of courtesy. I know this is the case sometimes also outside Japan. However, this practice is more and more often criticized, because it has been the cause of very embarrassing conflicts. Does JST maintain a "Ethics Committee" to study similar problems in the context of Japanese conditions ? I have anyway been relieved to read that one of the domestic reviewers thinks that "the overall impression on the results in this area is very satisfactory". I agree with the domestic reviewers to insist on the necessity of setting up a clear peer review system of evaluation; maybe in this case again recourse to foreign friends would be useful; with the possibility of doing it by e-mail, a very modest retribution would be suffcient. |
